Towards Universal Backward-Compatible Representation Learning

TL;DR

This paper introduces a universal backward-compatible training method that enables model upgrades in visual search systems without offline gallery refresh, effectively handling open-set scenarios and large-scale datasets.

Contribution

Proposes a novel universal backward-compatible training approach with a structural prototype refinement algorithm for all model upgrade scenarios.

Findings

Effective on large-scale face recognition datasets

Outperforms previous methods in open-set scenarios

Supports backfill-free model upgrades

Abstract

Conventional model upgrades for visual search systems require offline refresh of gallery features by feeding gallery images into new models (dubbed as "backfill"), which is time-consuming and expensive, especially in large-scale applications. The task of backward-compatible representation learning is therefore introduced to support backfill-free model upgrades, where the new query features are interoperable with the old gallery features. Despite the success, previous works only investigated a close-set training scenario (i.e., the new training set shares the same classes as the old one), and are limited by more realistic and challenging open-set scenarios. To this end, we first introduce a new problem of universal backward-compatible representation learning, covering all possible data split in model upgrades. We further propose a simple yet effective method, dubbed as Universal Backward-Compatible Training (UniBCT) with a novel structural prototype refinement algorithm, to learn compatible representations in all kinds of model upgrading benchmarks in a unified manner. Comprehensive experiments on the large-scale face recognition datasets MS1Mv3 and IJB-C fully demonstrate the effectiveness of our method.