ETran: Energy-Based Transferability Estimation

TL;DR

ETran introduces an energy-based metric for assessing the transferability of pre-trained models across various tasks, effectively identifying in-distribution and out-of-distribution datasets to improve model selection.

Contribution

This work is the first to propose transferability estimation for object detection and integrates energy-based models for a unified assessment across multiple tasks.

Findings

ETran outperforms previous methods by 21% on object detection benchmarks.

ETran achieves 12% improvement on classification benchmarks.

It provides state-of-the-art transferability assessment results.

Abstract

This paper addresses the problem of ranking pre-trained models for object detection and image classification. Selecting the best pre-trained model by fine-tuning is an expensive and time-consuming task. Previous works have proposed transferability estimation based on features extracted by the pre-trained models. We argue that quantifying whether the target dataset is in-distribution (IND) or out-of-distribution (OOD) for the pre-trained model is an important factor in the transferability estimation. To this end, we propose ETran, an energy-based transferability assessment metric, which includes three scores: 1) energy score, 2) classification score, and 3) regression score. We use energy-based models to determine whether the target dataset is OOD or IND for the pre-trained model. In contrast to the prior works, ETran is applicable to a wide range of tasks including classification, regression, and object detection (classification+regression). This is the first work that proposes transferability estimation for object detection task. Our extensive experiments on four benchmarks and two tasks show that ETran outperforms previous works on object detection and classification benchmarks by an average of 21% and 12%, respectively, and achieves SOTA in transferability assessment.