ALFA: Agglomerative Late Fusion Algorithm for Object Detection

TL;DR

ALFA is a new late fusion algorithm for object detection that uses agglomerative clustering to combine predictions from multiple detectors, improving accuracy on standard datasets.

Contribution

ALFA introduces a novel agglomerative clustering approach for late fusion in object detection, outperforming existing methods on PASCAL VOC datasets.

Findings

Achieves up to 32% lower error than best individual detectors.

Outperforms baseline fusion strategies and the DBF algorithm.

Effective with pairs and triplets of state-of-the-art detectors.

Abstract

We propose ALFA - a novel late fusion algorithm for object detection. ALFA is based on agglomerative clustering of object detector predictions taking into consideration both the bounding box locations and the class scores. Each cluster represents a single object hypothesis whose location is a weighted combination of the clustered bounding boxes. ALFA was evaluated using combinations of a pair (SSD and DeNet) and a triplet (SSD, DeNet and Faster R-CNN) of recent object detectors that are close to the state-of-the-art. ALFA achieves state of the art results on PASCAL VOC 2007 and PASCAL VOC 2012, outperforming the individual detectors as well as baseline combination strategies, achieving up to 32% lower error than the best individual detectors and up to 6% lower error than the reference fusion algorithm DBF - Dynamic Belief Fusion.

Figures

Tables

Table 1. TABLE I: Best cross-validated hyperparameter values

	Optimal values (SSD + DeNet)		Optimal values (FRCNN + SSD + DeNet)
Evaluation	mAP-s	mAP	mAP-s	mAP
Classification confidence	Multiplication	Averaging	Multiplication	Averaging
$\tau$	0.48	0.73	0.75	0.74
$\gamma$	0.22	0.25	0.28	0.30
$\epsilon$	0.56	0.26	0.17	0.39

Table 2. TABLE II: Results on PASCAL VOC 2007 and VOC 2012

Detector	fps (Hz)	PASCAL VOC 2007		PASCAL VOC 2012
		mAP-s (%)	mAP (%)	mAP-s (%)	mAP (%)
Faster R-CNN	7	77.95	78.83	72.72	73.59
SSD300	59	79.26	80.37	72.89	74.17
DeNet	33	78.09	79.26	70.73	72.10
SSD + DeNet
NMS	20.3	83.12	83.53	76.80	77.37
DBF	16.9	83.29	83.88	75.74	76.38
Fast ALFA	20.6	83.87	84.32	76.97	77.82
ALFA	18.1	84.16	84.41	77.52	77.98
SSD + DeNet + Faster R-CNN
NMS	5.2	84.31	84.43	78.11	78.34
DBF	4.7	84.97	85.24	75.71	75.69
Fast ALFA	5.2	85.78	85.67	79.16	79.42
ALFA	5.0	85.90	85.72	79.41	79.47

Table 3. TABLE III: Effects of various parameters on fusion performance

	PASCAL VOC 2007 test
Adding low-confidence class scores				✓	✓	✓	✓
Aggregating class scores		✓		✓	✓	✓	✓
Incorporate class scores into distance metric		✓	✓			✓	✓
Aggregating bounding boxes		✓	✓		✓		✓
ALFA (FRCNN + SSD + DeNet)	84.25	82.38	84.57	84.79	85.00	85.12	85.72

Equations

$$D_i(I)=\left\{(r_1,c_1),...,(r_{m_i},c_{m_i})\right\},i=1,...,N,$$

$$r=\left(x_{tl},y_{tl},x_{br},y_{br}\right),$$

$$0\le x_{tl}<x_{br}<I_{width},0\le y_{tl}<y_{br}<I_{height}.$$

$$c_j=\left(c_j^{(0)},c_j^{(1)},...,c_j^{(K)}\right),$$

$$\sum _{k=0}^K{c}^{(k)}=1,c^{(k)}\ge 0,k=0,...,K.$$

$$\sigma (C_i,C_j)=\underset{p\in C_i,\tilde{p}\in C_j}{\min }\sigma (p,\tilde{p}).$$

$$\underset{i,j}{\max }\sigma (C_i,C_j)<\tau ,$$

$$BC({\bar{c}}_i,{\bar{c}}_j)=\sum _{k=1}^K\sqrt{{\bar{c}}_i^{(k)}{\bar{c}}_j^{(k)}},$$

$${\bar{c}}^{(k)}=\frac{c^{(k)}}{1-c^{(0)}},k=1,...K.$$

$$IoU(r_i,r_j)=\frac{r_i\cap r_j}{r_i\cup r_j}.$$

$$\sigma (p_i,p_j)=IoU(r_i,r_j{)}^{\gamma }\cdot BC({\bar{c}}_i,{\bar{c}}_j{)}^{1-\gamma },$$

$$c_{lc}^{(i)}=c_{lc}^{(j)},i,j=1,...,K.$$

$$c_{lc}=\left(1-\epsilon ,\frac{\epsilon }{K},\frac{\epsilon }{K},...,\frac{\epsilon }{K}\right),$$

$$c_{\pi }=c_j,$$

$$j=\underset{i\in \pi }{\mathrm{argmax}}\underset{1\le k\le K}{\max }{c}_i^{(k)}.$$

$$c_{\pi }^{(k)}=\frac{1}{N}\left(\sum _{d=1}^s{c}_{i_d}^{(k)}+(N-s)\cdot c_{lc}^{(k)}\right),k=0,...,K.$$

$$c_{\pi }^{(k)}=\frac{{\tilde{c}}_{\pi }^{(k)}}{{\sum }_i{\tilde{c}}_{\pi }^{(i)}},$$

$${\tilde{c}}_{\pi }^{(k)}={\left(c_{lc}^{(k)}\right)}^{N-s}\prod _{d=1}^s{c}_{i_d}^{(k)},k=0,...,K.$$

$$r_{\pi }=r_j,$$

$$j=\underset{i\in \pi }{\mathrm{argmax}}\underset{1\le k\le K}{\max }{c}_i^{(k)}.$$

$$r_{\pi }=\frac{1}{|\pi |}\sum _{i\in \pi }{r}_i.$$

$$r_{\pi }=\sum _{i\in \pi }\frac{w_i}{{\sum }_{j\in \pi }{w}_j}\cdot r_i,$$

$$w_i=\underset{1\le k\le K}{\max }(c_i^{(k)}).$$

$$r_{\pi }=\frac{1}{{\sum }_{i\in \pi }{c}_i^{(l)}}\sum _{i\in \pi }{c}_i^{(l)}\cdot r_i,$$

$$l=\underset{k\ge 1}{\mathrm{argmax}}{c}_{\pi }^{(k)}.$$

$$c^{(l)}\ge \theta ,$$

$${\alpha }_i^{(k)}=n_i^{(k)}/n^{(k)},$$

$$A{P}^{(k)}=\sum _i{\alpha }_i^{(k)}A{P}_i^{(k)}.$$

$$\text{mbox}mAP=\frac{1}{K}\sum _{k=1}^K\text{mbox}{AP}^{(k)}.$$

Full text

ALFA was evaluated using combinations of a pair (SSD and DeNet) and a triplet (SSD, DeNet and Faster R-CNN) of recent object detectors that are close to the state-of-the-art. ALFA achieves state of the art results on PASCAL VOC 2007 and PASCAL VOC 2012, outperforming the individual detectors as well as baseline combination strategies, achieving up to 32% lower error than the best individual detectors and up to 6% lower error than the reference fusion algorithm DBF – Dynamic Belief Fusion.

I Introduction

Object detection is an important and challenging task in computer vision with a lot of applications. In recent years accuracy of object detectors significantly improved due to use of learning. R-CNN [6] was the first breakthrough associated with using deep convolutional neural networks for object detection. Fast R-CNN [5] and Faster R-CNN [14] develop the idea further improving both detection speed and accuracy. You Only Look Once [12], Single-Shot Detector [11] introduced more lightweight approach and end-to-end training achieving remarkable accuracy while operating in real-time. YOLOv2 [13] and DSSD [4] and most recent DeNet [16] object detector push these boundaries even further.

It is known that aggregating machine learning algorithms in an ensemble tend to improve performance [1]. Aggregating different object detectors usually called fusion was shown to improve accuracy as well. So called late fusion methods treat base object detectors as black boxes using only their predictions as input.

This paper studies the problem of late fusion on object detector outputs. We want to explore whether modern deep object detectors differ from each other enough so their fusion could show significant performance boost in comparison to individual detectors.

The problem of object detector fusion has been addressed in the literature. Detect2Rank [8] uses Learn2Rank algorithms to rank detections from different object detectors. This goal is achieved by representing each detection with a feature vector and learning a ranking system on a validation set. Handcrafted feature vector includes information about detector-detector context, object saliency and object-object relation information. Ranking is learned using L2 regularized support vector classifier, logistic regressor, support vector regressor and RankSVM. Non-maximum suppression is then used to remove multiple detections of the same object with lower scores.

Dynamic Belief Fusion is a late fusion algorithm that re-scores detections confidence using Dempster-Shafer theory [9]. DBF solves object detection as a binary classification problem – each bounding box either contains an object or not. Using precision-recall curves built for each base detector and for abstract hyperparameter-dependent “best possible detectors” all detections are re-scored and non-maximum suppression is applied. Authors of [9] claim that DBF outperforms all existing fusion methods including Detect2Rank. Unfortunately, the paper does not go into much detail regarding extending binary DBF framework to multiclass object detection scenario.

Faster R-CNN with ResNet-101 as a base network currently acheive state of the art results in object detection in terms of mAP while being quite slow. These object detectors were used as base detectors in an ensemble in [7]. Several Region Proposal Networks and Classification Networks were trained independently, at test time union set of region predictions from all RPNs is classified by an ensemble of classification networks. Non-maximum suppression is run afterwards. Using three models in an ensemble boosts performance from 34.9 mAP to 37.4 mAP on MS COCO object detection dataset [10].

None of these methods provides new bounding boxes as fusion outputs, bounding box for the object is always one of the bounding boxes predicted by base detectors. We believe that combining bounding boxes from all base detectors can lead to better object localization.

I-A Contribution

Our contribution is as follows:

• •

We show that modern deep object detectors differ enough so their combination significantly outperforms individual detectors.

• •

We propose novel Agglomerative Late Fusion Algorithm (ALFA) for object detection that shows state of the art results on PASCAL VOC 2007 [2] and PASCAL VOC 2012 [3] object detection datasets reducing error by up to 32% in comparison with individual detectors and by up to 6% in comparison with the reference fusion methods.

• •

We clarify DBF extension to multiclass object detection scenario and provide experimental results for modern detectors fusion using DBF.

• •

We make source code for ALFA and our implementation of DBF publicly available making results of our research reproducible: http://github.com/IuliiaSaveleva/ALFA

II Base Object Detectors

Assume $N$ base object detectors $D_{1},D_{2},...,D_{N}$ for $K$ classes. After processing image $I$, detector $i$ outputs $m_{i}$ predictions of object presence. Each prediction consists of bounding box coordinates and a $(K+1)$-tuple of class scores:

[TABLE]

where $r_{j}$ are the four coordinates of the axis-aligned bounding box and $c_{j}$ are the class scores for the $j$-th detected object.

[TABLE]

where $(x_{tl},y_{tl})$ are the coordinates of the top-left corner of the bounding box and $(x_{br},y_{br})$ are the coordinates of the bottom-right corner of the bounding box and

[TABLE]

Class scores $c_{j}$ is a tuple

[TABLE]

where $c_{j}^{(0)}$ is the “no object” probability and $c_{j}^{(1)},c_{j}^{(2)},...,c_{j}^{(K)}$ are the probabilities for $K$ classes,

[TABLE]

The core problem for fusion of multiple detectors is the decision which detections are due to the same object.

III The Proposed Method

We formulate the problem of late fusion as agglomerative clustering and propose a parametrized similarity function that takes into account both spatial properties of the two predictions and their class scores. Parameters are learned on validation set. We define object proposal to be the set of predictions forming a cluster. A proposal therefore comprises one or more predictions.

Assuming that predictions with similar bounding boxes and class scores often correspond to the same object, only one bounding box and class score tuple is output per proposal. We explore several strategies for estimating the object proposal bounding box and classification. We employ non-maximum suppression with IoU threshold 0.5 to remove remaining multiple detections of the same object.

III-A The Clustering Method

To define clustering procedure one needs to define similarity score function between two samples, similarity score functions between two clusters and stopping criteria.

III-A1 Similarity Between Proposals

We assume that all predictions associated with the object proposal are due to the same object. Hence, each prediction bounding box and class scores should be similar to each other prediction bounding box and class scores. This intuition strongly suggests using complete-link clustering. Let $C_{i}$ and $C_{j}$ be two clusters and $\sigma(p,\tilde{p})$ – similarity score function between predictions $p$ and $\tilde{p}$ that will be defined later. We define the following similarity score function for prediction clusters:

[TABLE]

III-A2 Stopping Criteria

Clustering stops when

[TABLE]

where $\tau$ is a hyperparameter. Therefore, every two predictions associated with the same object proposal have similarity score no less than $\tau$.

III-A3 Similarity Between Predictions

We assume that detections with substantially different class scores often correspond to different objects on the image. Therefore, we want to incorporate class scores similarity into our similarity metric. We employ Bhattacharyya coefficient as a measure of similarity between class scores:

[TABLE]

where $\bar{c}$ is obtained from class score tuple $c$ by omitting the zeroth “no object” component and renormalizing:

[TABLE]

Note that $BC(\bar{c}_{i},\bar{c}_{j})\in[0,1]$ and equals 1 if and only if $\bar{c}_{i}=\bar{c}_{j}$.

Two detections with similar shape, size and position on the image will often correspond to the same object. To decide whether two detections correspond to the same object we should account for similarity between their bounding boxes. We use intersection over union coefficient which is widely used as a measure of similarity between bounding boxes:

[TABLE]

We propose the following measure of similarity between object detectors predictions $p_{i}=(r_{i},c_{i})$ and $p_{j}=(r_{j},c_{j})$ that takes into account both class scores similarity and similarity of bounding boxes:

[TABLE]

where $\gamma\in[0,1]$ is a hyperparameter. Note that $\sigma(p_{i},p_{j})\in[0,1]$, $\sigma(p_{i},p_{j})$ equals 0 if bounding boxes for predictions $p_{i}$ and $p_{j}$ do not intersect or their class scores do not overlap. With $\gamma\in(0,1)$, $\sigma(p_{i},p_{j})=1$ if and only if bounding boxes and class scores for $p_{i}$ and $p_{j}$ are equal correspondingly.

Since NMS is applied to every base detector predictions before the fusion procedure, we assume that multiple similar detections from the same base detector are often due to different objects. Therefore, detections from the same base detector should be assigned to different clusters. To achieve that, we set similarity scores between predictions from the same base detector to zero.

III-B Decision on the class of the cluster

Assume predictions from detectors $D_{i_{1}},D_{i_{2}},...,D_{i_{s}}$ were assigned to object proposal $\pi$. Confidence of the prediction is a score value of the predicted class. Object detector fusion is not straightforward because of variable number of predictions corresponding to each object proposal.

To account for missing detections associated with an object proposal we assign an additional low-confidence class scores tuple to this object proposal for every detector that missed. Low-confidence class scores should not influence class prediction for the object proposal, therefore, last $K$ components of a tuple should be equal:

[TABLE]

Moreover, since recall of modern detectors is less than 1, missed detection does not always mean that there is no object. We use the following low-confidence class scores:

[TABLE]

where $\varepsilon$ is a hyperparameter. Assigning low-confidence class scores to object proposal significantly lowers the confidence of the resulting predictions if $\varepsilon$ is close to zero.

Possible strategies for dealing with class scores aggregation are inspired by classification ensembles and include:

• •

Choosing most confident prediction associated with the object proposal:

[TABLE]

where

[TABLE]

• •

Averaging fusion. Averaging class scores vectors of associated predictions:

[TABLE]

• •

Multiplication fusion. Multiplying class scores corresponding to the same class and renormalizing:

[TABLE]

where

[TABLE]

III-C Object localization

We have explored several object localization strategies:

• •

Bounding box associated with the most confident prediction. Choosing bounding box corresponding to the most confident prediction associated with the object proposal:

[TABLE]

where

[TABLE]

This principle is usually used in non-maximum suppression algorithms. NMS is often applied as a final step in fusion algorithms.

• •

Averaged bounding box. Averaging outputs of the weak learners is a common way to aggregate predictions to produce output of an ensemble:

[TABLE]

• •

Bounding box obtained by weighted averaging. Correct object localization usually associated with more confident prediction. Following this intuition we assign weights to bounding boxes based on prediction confidence. Fusion outputs the following bounding box for object proposal $\pi$:

[TABLE]

where

[TABLE]

• •

Average weighted by proposal class confidence This localization strategy is similar to the previous one except we use class scores of the label that is chosen for the object proposal as weights:

[TABLE]

where

[TABLE]

This strategy relies on the assumption that localization of an object depends on the predicted class scores.

IV Experiments

We evaluated ALFA and baseline methods on PASCAL VOC 2007 and VOC 2012 object detection datasets using the following base object detectors:

• •

Single Shot Detector: SSD-300 with VGG-16 as a base network trained on PASCAL VOC 07+12 trainval dataset, i.e. on the union of the training and validation images of the PASCAL VOC 2007 and VOC 2012 datasets.

• •

DeNet with skip layers and ResNet-101 as a base network trained on PASCAL VOC 07+12 trainval dataset.

• •

Faster R-CNN with ResNet-101 as a base network trained on PASCAL VOC 07+12 trainval dataset. We do not use global context, multi-scale testing and bounding box refinement introduced in [7].

These object detection methods were selected for three reasons. First, none of them is slow and its combination that requires running all three is not impractical. Second, these are commonly used object detectors with performance no far from the state of the art. Third, for SSD and DeNet the source code and the weights for the object detector are available from the authors.

IV-A Base Detectors

Faster R-CNN [14] uses Region Proposal Network which takes last feature map as an input and outputs a number of bounding boxes that could potentially contain objects. The second component – Fast R-CNN – is used to determine whether patricular proposed bounding box contains an object of certain class or not. While in the original paper VGG [15] is used as base network, it was shown in [7] that using ResNet-101 as base network improves accuracy significantly. Faster R-CNN with ResNet-101 base network with multi-scale testing, inclusion of context and bounding box refinement, sometimes referred to as Faster R-CNN+++, shows state of the art performance in terms of accuracy but is slower than YOLO, SSD and DeNet.

Single Shot Detector [11] does not use RPN but instead relies on convolutional detectors to classify objects on different scales. SSD uses VGG convolutional network with additional convolutional layers for feature extraction. Convolutional detectors for different feature maps predict object classes and offsets with respect to corresponding default bounding boxes. Making use of feature maps at different depths allows SSD to detect objects with different sizes. Neurons from not-too-deep layers of CNN have smaller receptive fields thus are suitable for small objects detection. Neurons at deeper layers have larger receptive fields that allow detection of bigger objects. Convolutional detector consists of $(K+1)+4$ convolutions with $3\times 3\times d$ filter size predicting class scores for $K$ classes along with ”no object” class and offsets of the bounding box with respect to associated with the detector default bounding box. Non-maximum suppression is applied to all the predictions from convolutional detectors at each scale.

DeNet [16] relies on region-of-interest estimation based on corner detection. DeNet predicts for each image pixel how likely it is a certain corner of an object bounding box. Predicted corners form regions-of-interest that are classified by neural network. Therefore, DeNet uses neither Region Proposal Network nor handcrafted default boxes for object localization. ResNet-101 [7] is used as a base network, deconvolutions are applied to the last layer of ResNet in order to increase localization accuracy. DeNet acheives remarkable performance in terms of speed and accuracy outperforming SSD.

IV-B Baseline methods

IV-B1 Dynamic Belief Fusion

We use Dynamic Belief Fusion as a baseline since it was shown to outperform other fusion methods [9]. We use our own implementation of DBF since source code for DBF is not available.

Dynamic Belief Fusion [9] relies on precision-recall curves, thus, binary decision problem is assumed. While [9] clearly states that DBF is applied to object detection on multiclass PASCAL VOC 2007 dataset, only binary procedure is described in the paper and it is not clear how to extend DBF to multiclass scenario. We tested three options how to handle multiclass case in DBF:

Object detection is treated as a binary problem – object is either present in a bounding box or not. In this case each detector has one class-agnostic precision-recall curve. We do not aggregate bounding boxes with different labels so decision on a final label is trivial. 2. 2.

Each base detector is considered as $K$ class-specific binary object detectors and every detection with class scores $c$ is treated as $K$ detections with confidence values $c^{(1)},....,c^{(K)}$. Each base detector has $K$ class-specific precision-recall curves associated with it. We also assume $K$ class-specific “best possible detectors”. 3. 3.

Each base object detector is again treated as $K$ class-specific binary object detectors with $K$ class-specific precision-recall curves but this time there is only one class-agnostic “best possible detector”.

We have implemented all three versions and measured their performance. We use the third variant as a baseline since it outperforms the first and the second option.

IV-B2 Non-Maximum Suppression

We use Greedy Non-Maximum Suppression as our second baseline since it was successfully used in [7] to aggregate object detectors outputs. GreedyNMS suppresses every prediction that overlaps with $\mbox{IoU}>0.5$ with more confident prediction of the same class.

IV-C Note on confidence thresholds

It is possible to set confidence threshold of any base detector to exclude least confident detections.

We can set threshold $\theta$ for an object detector and consider only those predictions with class scores $c$ for which the following holds:

[TABLE]

where $l$ is the predicted label.

Value of $\theta$ for base detectors affects fusion performance. Bigger $\theta$ results in decreased number of predictions speeding up fusion algorithm since amount of computation is proportional to the number of predictions.

We choose $\theta$ for DBF and NMS to maximize fusion mAP. For ALFA we use two thresholds chosen to achieve two different goals: (i) maximize fusion mAP and (ii) maximize mAP while keeping fusion computation under 2-3 ms. We refer to this version as “Fast ALFA” in experiment results. Fusion performance is measured on Intel Core i5-6400.

For all tested fusion algorithms smaller values of $\theta$ result in higher mAP and lower speed performance. We use $\theta=0.015$ for NMS, DBF and ALFA. We use $\theta=0.05$ for Fast ALFA.

IV-D Note on evaluation procedure

Object detector outputs a class scores vector for every detection. There are two ways of handling obtained class scores vectors:

• •

Consider every detection with class scores vector $c$ and bounding box $r$ to be a single detection with label $l$, where $l=\operatorname*{argmax}_{1\leq k\leq K}c^{(k)}$, confidence $c^{(l)}$ and bounding box $r$. We refer to mean average precision computed this way as mAP-s.

• •

Treat every detection with class scores vector $c$ and bounding box $r$ as multiple detections sharing the same bounding box – one detection for each label $l$, where $l=1,...,K$, with confidence value $c^{(l)}$ and bounding box $r$.

Since this approach is more common it is referred to as mAP in experiments results.

IV-E Results on PASCAL VOC 2007

We employ 5-fold cross-validation on PASCAL VOC 2007 test to choose optimal lozalization and recognition strategies and to adjust hyperparameters $\gamma$, $\tau$ and $\varepsilon$. Best performing localization strategy is averaging weighted by proposal class confidence. Best hyperparameter values obtained through cross-validation are listed in Table I.

Precision-recall curves for DBF are also estimated using 5-fold cross-validation.

During 5-fold cross-validation mean average precision is computed as follows:

• •

We compute average precision $AP_{i}^{(k)}$ on the $i$-th test part, $i=1,...,5$, for each class $k$, $k=1,...,K$.

• •

We compute weights for each class for the $i$-th test part:

[TABLE]

where $n_{i}^{(k)}$ is the number of objects of class $k$ in the $i$-th test part and $n^{(k)}$ is the number of objects of class $k$ in the whole test set.

• •

Average precision for each class is computed as follows:

[TABLE]

• •

Mean average precision:

[TABLE]

This procedure introduces small bias. To make the comparison fair we use the same mAP computation procedure for all individual detectors and fusion methods on PASCAL VOC 2007.

Overall mAP and detection speed are provided in Table II.

IV-F Results on PASCAL VOC 2012

We also evaluated all reviewed fusion methods performance on PASCAL VOC 2012 test set. Hyperparameters of ALFA and DBF were adjusted on PASCAL VOC 2007 test set. Results are presented in Table II. Surprisingly, DBF shows worse results than NMS in this scenario. We think that DBF does not generalize well across datasets with different difficulty since it relies on precision-recall curves.

IV-G Ablation study

The influence of the following design decisions is measured:

• •

Adding low-confidence class scores for every missed detection.

• •

Aggregating class scores instead of using the most confident prediction. Adding low-confidence detections do not affect performance when using class scores for most confident prediction.

• •

Taking class scores into account while generating object proposals.

• •

Using weighted average bounding box instead of using bounding box of the most confident prediction.

Results of the ablation study are summarized in Table III.

V Discussion

ALFA shows higher mean average precision values for each detector combination on both PASCAL VOC 2007 and VOC 2012 when compared with base detectors and baseline fusion methods. Fast ALFA is slightly inferior to ALFA but still outperforming baseline fusion methods while being marginally faster.

To achieve close-to-real-time performance fusion method needs not only very fast base object detectors but also computationally light aggregation procedure. The key to fast implementation of ALFA is to break all objects into groups so that each object in the group is similar to at least one another object in this group. This step can be done quite effectively. Agglomerative clustering is then applied to each group independently. Computation time for our fusion procedure is as low as 1.2 ms for two detectors and 1.6 ms for three detectors (for $\theta=0.05$) while code is written in Python.

It is reasonable to assume, however, that ALFA performance will be slower for images crowded with objects from the same class with significantly overlapping bounding boxes.

VI Conclusion

We propose ALFA – a novel late fusion algorithm for object detection. ALFA shows state of the art results on PASCAL VOC 2007 and VOC 2012 datasets outperforming individual detectors and existing fusion frameworks regardless of evaluation procedure while being computationally light. The classification error expressed as $1-\mbox{\it mAP}$, is reduced by up to 32% in comparison to the base detectors and by up to 6% when compared with the state of the art fusion method DBF, that, as experiments indicate, does not generalize well across different datasets.

Acknowledgment

E. Razinkov was funded by the Russian Government support of the Program of Competitive Growth of Kazan Federal University among World’s Leading Academic Centers and by Russian Foundation of Basic Research, project number 16-01-00109a. J. Matas was supported by Czech Science Foundation Project GACR P103/12/G084.