ConR: Contrastive Regularizer for Deep Imbalanced Regression
Mahsa Keramati, Lili Meng, R. David Evans
TL;DR
ConR is a contrastive regularizer designed to improve deep imbalanced regression by modeling label similarities and preventing feature collapse of minority samples, significantly enhancing existing methods.
Contribution
It introduces a novel contrastive regularizer that effectively handles continuous label spaces and imbalances, extending to multi-dimensional labels and integrating easily with existing models.
Findings
ConR significantly improves performance on imbalanced regression benchmarks.
It effectively models local and global label similarities.
The method is compatible with various existing approaches.
Abstract
Imbalanced distributions are ubiquitous in real-world data. They create constraints on Deep Neural Networks to represent the minority labels and avoid bias towards majority labels. The extensive body of imbalanced approaches address categorical label spaces but fail to effectively extend to regression problems where the label space is continuous. Local and global correlations among continuous labels provide valuable insights towards effectively modelling relationships in feature space. In this work, we propose ConR, a contrastive regularizer that models global and local label similarities in feature space and prevents the features of minority samples from being collapsed into their majority neighbours. ConR discerns the disagreements between the label space and feature space and imposes a penalty on these disagreements. ConR addresses the continuous nature of label space with two main…
Peer Reviews
Decision·ICLR 2024 poster
1. Data imbalance and the fairness of machine learning algorithms are practical issues that warrant significant attention. 2. The method is reasonably designed and has shown good experimental results.
1. While the paper presents a contrastive learning paradigm adapted for regression, it does not appear to directly address the issue of data imbalance. It would enhance the paper if the authors could clarify how the method specifically tackles this challenge or consider adapting the technique to more explicitly focus on imbalanced datasets. 2. Comparisons should be made with other contrastive regression learning methods (e.g., [a]). 3. The manuscript could be strengthened by providing some theor
1. This paper is well-written and easy to understand. 2. The novelty of this paper is good. To my knowledge, applying contrastive learning to imbalance regression is novel. 3. The authors provide a new dataset with 2-dimentional label space by using MPIIGaze, which could be useful to the imbalance regression community.
1. The proposed objective in Eq. (4) is relative hasty, without the reason of introducing the ordinary regression loss. Also, the definition of the introduced loss should be clearly provided. 2. No deviation measure in the experimental results. 3. No experimental result for the ConR-only case. All the results for the proposed ConR are with respect to the combination with existing methods as a regularized. Due to supervised information is already used in the loss of ConR, its preformation should
1. ConR presents a novel auggmentation approach to handle the imbalanced problem in continuous label spaces, whose problem is important. 2. The methods of regularizing process of **ConR** by pulling together positive pairs and relatively repelling negative pairs seems solid. 3. The results seem to indicate that the proposed method can be seamlessly integrated with other models and exhibits improvements over existing baselines.
1. While the author claims that **ConR** reduces prediction error, are there any theoretical insights or guarantees supporting the idea that **ConR** can achieve a lower generalization bound? Relying solely on empirical results might not suffice to attest to the superiority of the proposed methods. 2. I am curious about complexity. When performing augmentation on large-scale datasets, sampling might increase the complexity. This leads to a prevalent question: Why not leverage reweighting method
Code & Models
Videos
Taxonomy
TopicsText and Document Classification Technologies · Imbalanced Data Classification Techniques · Machine Learning and Data Classification
Methodsfail