# A Tunable Loss Function for Binary Classification

## TL;DR

This paper introduces a tunable loss function called $oldsymbol{ extalpha}$-loss for binary classification, bridging log-loss and 0-1 loss, with theoretical properties and empirical performance improvements.

## Contribution

The paper proposes $oldsymbol{ extalpha}$-loss, a new flexible loss function with proven calibration and margin properties, and demonstrates its effectiveness in logistic regression on MNIST.

## Key findings

- $oldsymbol{ extalpha}$-loss is classification-calibrated and has an equivalent margin form.
- An upper bound on risk difference is derived for $oldsymbol{ extalpha}$-loss in logistic regression.
- $oldsymbol{ extalpha=2}$ performs better than log-loss on MNIST.

## Abstract

We present $\alpha$-loss, $\alpha \in [1,\infty]$, a tunable loss function for binary classification that bridges log-loss ($\alpha=1$) and $0$-$1$ loss ($\alpha = \infty$). We prove that $\alpha$-loss has an equivalent margin-based form and is classification-calibrated, two desirable properties for a good surrogate loss function for the ideal yet intractable $0$-$1$ loss. For logistic regression-based classification, we provide an upper bound on the difference between the empirical and expected risk at the empirical risk minimizers for $\alpha$-loss by exploiting its Lipschitzianity along with recent results on the landscape features of empirical risk functions. Finally, we show that $\alpha$-loss with $\alpha = 2$ performs better than log-loss on MNIST for logistic regression.