Dist Loss: Enhancing Regression in Few-Shot Region through Distribution Distance Constraint

TL;DR

This paper introduces Dist Loss, a novel differentiable loss function that minimizes distribution distance between predictions and labels, improving regression performance in imbalanced, few-shot data scenarios across multiple domains.

Contribution

The paper proposes Dist Loss, a new loss function that incorporates distribution information into regression training, specifically enhancing performance in imbalanced, few-shot regions.

Findings

Dist Loss improves regression accuracy in few-shot regions.

It achieves state-of-the-art results on diverse datasets.

Easy to integrate with existing models.

Abstract

Imbalanced data distributions are prevalent in real-world scenarios, posing significant challenges in both imbalanced classification and imbalanced regression tasks. They often cause deep learning models to overfit in areas of high sample density (many-shot regions) while underperforming in areas of low sample density (few-shot regions). This characteristic restricts the utility of deep learning models in various sectors, notably healthcare, where areas with few-shot data hold greater clinical relevance. While recent studies have shown the benefits of incorporating distribution information in imbalanced classification tasks, such strategies are rarely explored in imbalanced regression. In this paper, we address this issue by introducing a novel loss function, termed Dist Loss, designed to minimize the distribution distance between the model's predictions and the target labels in a differentiable manner, effectively integrating distribution information into model training. Dist Loss enables deep learning models to regularize their output distribution during training, effectively enhancing their focus on few-shot regions. We have conducted extensive experiments across three datasets spanning computer vision and healthcare: IMDB-WIKI-DIR, AgeDB-DIR, and ECG-Ka-DIR. The results demonstrate that Dist Loss effectively mitigates the negative impact of imbalanced data distribution on model performance, achieving state-of-the-art results in sparse data regions. Furthermore, Dist Loss is easy to integrate, complementing existing methods.