Detection Hub: Unifying Object Detection Datasets via Query Adaptation on Language Embedding

TL;DR

Detection Hub introduces a dataset-aware, category-aligned approach that unifies multiple object detection datasets using language embeddings, significantly improving detection performance and achieving state-of-the-art results.

Contribution

The paper proposes a novel Detection Hub framework that addresses dataset inconsistency in object detection by learning dataset embeddings and aligning categories via language embeddings.

Findings

Joint training on multiple datasets improves detection accuracy.

Detection Hub achieves state-of-the-art results on UODB benchmark.

Semantic category alignment enhances cross-dataset learning.

Abstract

Combining multiple datasets enables performance boost on many computer vision tasks. But similar trend has not been witnessed in object detection when combining multiple datasets due to two inconsistencies among detection datasets: taxonomy difference and domain gap. In this paper, we address these challenges by a new design (named Detection Hub) that is dataset-aware and category-aligned. It not only mitigates the dataset inconsistency but also provides coherent guidance for the detector to learn across multiple datasets. In particular, the dataset-aware design is achieved by learning a dataset embedding that is used to adapt object queries as well as convolutional kernels in detection heads. The categories across datasets are semantically aligned into a unified space by replacing one-hot category representations with word embedding and leveraging the semantic coherence of language embedding. Detection Hub fulfills the benefits of large data on object detection. Experiments demonstrate that joint training on multiple datasets achieves significant performance gains over training on each dataset alone. Detection Hub further achieves SoTA performance on UODB benchmark with wide variety of datasets.