On the Use of Anchoring for Training Vision Models

TL;DR

This paper investigates anchoring as a training principle for vision models, identifies a key problem with undesirable shortcuts, and proposes a regularized protocol that improves generalization and safety across diverse datasets and architectures.

Contribution

It introduces a regularized anchored training protocol that mitigates shortcut learning and enhances generalization in vision models.

Findings

Regularized anchored training improves generalization performance.

The new protocol reduces learning undesirable shortcuts.

Empirical results show enhanced safety metrics.

Abstract

Anchoring is a recent, architecture-agnostic principle for training deep neural networks that has been shown to significantly improve uncertainty estimation, calibration, and extrapolation capabilities. In this paper, we systematically explore anchoring as a general protocol for training vision models, providing fundamental insights into its training and inference processes and their implications for generalization and safety. Despite its promise, we identify a critical problem in anchored training that can lead to an increased risk of learning undesirable shortcuts, thereby limiting its generalization capabilities. To address this, we introduce a new anchored training protocol that employs a simple regularizer to mitigate this issue and significantly enhances generalization. We empirically evaluate our proposed approach across datasets and architectures of varying scales and complexities, demonstrating substantial performance gains in generalization and safety metrics compared to the standard training protocol.