Cardinality augmented loss functions

TL;DR

This paper introduces cardinality augmented loss functions that leverage mathematical invariants to improve neural network training on imbalanced datasets, enhancing minority class performance and overall accuracy.

Contribution

It proposes a novel class of loss functions based on cardinality-like invariants to address class imbalance in neural network training.

Findings

Significant improvement in minority class accuracy

Enhanced overall performance metrics

Effective on both artificial and real-world datasets

Abstract

Class imbalance is a common and pernicious issue for the training of neural networks. Often, an imbalanced majority class can dominate training to skew classifier performance towards the majority outcome. To address this problem we introduce cardinality augmented loss functions, derived from cardinality-like invariants in modern mathematics literature such as magnitude and the spread. These invariants enrich the concept of cardinality by evaluating the `effective diversity' of a metric space, and as such represent a natural solution to overly homogeneous training data. In this work, we establish a methodology for applying cardinality augmented loss functions in the training of neural networks and report results on both artificially imbalanced datasets as well as a real-world imbalanced material science dataset. We observe significant performance improvement among minority classes, as well as improvement in overall performance metrics.