Transferability Metrics for Object Detection

TL;DR

This paper extends transferability metrics to object detection, introducing TLogME with local feature extraction, which effectively predicts transfer performance and outperforms existing metrics across various datasets and models.

Contribution

The authors develop a novel transferability metric, TLogME, tailored for object detection, incorporating local features and coordinate regression, advancing transferability estimation beyond classification.

Findings

TLogME shows robust correlation with transfer performance across tasks.

TLogME outperforms existing transferability metrics in experiments.

Local feature extraction improves transferability prediction accuracy.

Abstract

Transfer learning aims to make the most of existing pre-trained models to achieve better performance on a new task in limited data scenarios. However, it is unclear which models will perform best on which task, and it is prohibitively expensive to try all possible combinations. If transferability estimation offers a computation-efficient approach to evaluate the generalisation ability of models, prior works focused exclusively on classification settings. To overcome this limitation, we extend transferability metrics to object detection. We design a simple method to extract local features corresponding to each object within an image using ROI-Align. We also introduce TLogME, a transferability metric taking into account the coordinates regression task. In our experiments, we compare TLogME to state-of-the-art metrics in the estimation of transfer performance of the Faster-RCNN object detector. We evaluate all metrics on source and target selection tasks, for real and synthetic datasets, and with different backbone architectures. We show that, over different tasks, TLogME using the local extraction method provides a robust correlation with transfer performance and outperforms other transferability metrics on local and global level features.