Wrapped Loss Function for Regularizing Nonconforming Residual Distributions

TL;DR

This paper introduces a wrapped loss function to address nonconforming residual distributions in multi-output machine learning, leading to faster convergence, improved accuracy, and better handling of imbalanced data.

Contribution

The paper proposes a novel wrapped loss function that preserves the original loss's gradient while improving convergence and accuracy in multi-output learning.

Findings

Faster convergence compared to standard loss functions

Enhanced accuracy in multi-output tasks

Improved performance on imbalanced datasets

Abstract

Multi-output is essential in machine learning that it might suffer from nonconforming residual distributions, i.e., the multi-output residual distributions are not conforming to the expected distribution. In this paper, we propose "Wrapped Loss Function" to wrap the original loss function to alleviate the problem. This wrapped loss function acts just like the original loss function that its gradient can be used for backpropagation optimization. Empirical evaluations show wrapped loss function has advanced properties of faster convergence, better accuracy, and improving imbalanced data.