ACCon: Angle-Compensated Contrastive Regularizer for Deep Regression

TL;DR

This paper introduces ACCon, a novel angle-compensated contrastive regularizer for deep regression that improves model performance, especially on imbalanced datasets and with limited data, by leveraging a new contrastive learning approach.

Contribution

It proposes a new contrastive regularizer that adjusts cosine distances based on label-representation relationships, extending contrastive learning to regression tasks.

Findings

Improves regression accuracy on imbalanced datasets.

Enhances data efficiency and robustness with limited samples.

Achieves competitive performance compared to existing methods.

Abstract

In deep regression, capturing the relationship among continuous labels in feature space is a fundamental challenge that has attracted increasing interest. Addressing this issue can prevent models from converging to suboptimal solutions across various regression tasks, leading to improved performance, especially for imbalanced regression and under limited sample sizes. However, existing approaches often rely on order-aware representation learning or distance-based weighting. In this paper, we hypothesize a linear negative correlation between label distances and representation similarities in regression tasks. To implement this, we propose an angle-compensated contrastive regularizer for deep regression, which adjusts the cosine distance between anchor and negative samples within the contrastive learning framework. Our method offers a plug-and-play compatible solution that extends most existing contrastive learning methods for regression tasks. Extensive experiments and theoretical analysis demonstrate that our proposed angle-compensated contrastive regularizer not only achieves competitive regression performance but also excels in data efficiency and effectiveness on imbalanced datasets.