Bipartite Mode Matching for Vision Training Set Search from a Hierarchical Data Server

TL;DR

This paper introduces a hierarchical data server and bipartite mode matching algorithm to select training data that better aligns with target domain modes, improving model performance in vision tasks without real-time annotation.

Contribution

The paper proposes a novel hierarchical data server structure combined with bipartite mode matching for optimized training set selection, enhancing domain adaptation in vision models.

Findings

Matched server modes have smaller domain gaps with target data.

Models trained on matched sets outperform baseline methods.

Combining BMM with existing UDA methods yields further accuracy improvements.

Abstract

We explore a situation in which the target domain is accessible, but real-time data annotation is not feasible. Instead, we would like to construct an alternative training set from a large-scale data server so that a competitive model can be obtained. For this problem, because the target domain usually exhibits distinct modes (i.e., semantic clusters representing data distribution), if the training set does not contain these target modes, the model performance would be compromised. While prior existing works improve algorithms iteratively, our research explores the often-overlooked potential of optimizing the structure of the data server. Inspired by the hierarchical nature of web search engines, we introduce a hierarchical data server, together with a bipartite mode matching algorithm (BMM) to align source and target modes. For each target mode, we look in the server data tree for the best mode match, which might be large or small in size. Through bipartite matching, we aim for all target modes to be optimally matched with source modes in a one-on-one fashion. Compared with existing training set search algorithms, we show that the matched server modes constitute training sets that have consistently smaller domain gaps with the target domain across object re-identification (re-ID) and detection tasks. Consequently, models trained on our searched training sets have higher accuracy than those trained otherwise. BMM allows data-centric unsupervised domain adaptation (UDA) orthogonal to existing model-centric UDA methods. By combining the BMM with existing UDA methods like pseudo-labeling, further improvement is observed.