Margin and Consistency Supervision for Calibrated and Robust Vision Models

TL;DR

MaCS is a regularization framework that improves calibration and robustness of vision models by enforcing logit margins and local prediction stability, applicable across architectures without extra data or overhead.

Contribution

Introduces MaCS, a simple, architecture-agnostic regularization method combining margin and consistency losses to enhance calibration and robustness in vision models.

Findings

MaCS improves calibration metrics like ECE and NLL.

MaCS enhances robustness to common corruptions.

MaCS maintains or boosts top-1 accuracy.

Abstract

Deep vision classifiers often achieve high accuracy while remaining poorly calibrated and fragile under small distribution shifts. We present Margin and Consistency Supervision (MaCS), a simple, architecture-agnostic regularization framework that jointly enforces logit-space separation and local prediction stability. MaCS augments cross-entropy with (i) a hinge-squared margin penalty that enforces a target logit gap between the correct class and the strongest competitor, and (ii) a consistency regularizer that minimizes the KL divergence between predictions on clean inputs and mildly perturbed views. We provide a unifying theoretical analysis showing that increasing classification margin while reducing local sensitivity formalized via a Lipschitz-type stability proxy yields improved generalization guarantees and a provable robustness radius bound scaling with the margin-to-sensitivity ratio. Across several image classification benchmarks and several backbones spanning CNNs and Vision Transformers, MaCS consistently improves calibration (lower ECE and NLL) and robustness to common corruptions while preserving or improving top-1 accuracy. Our approach requires no additional data, no architectural changes, and negligible inference overhead, making it an effective drop-in replacement for standard training objectives.