A novel multi-scale loss function for classification problems in machine learning

TL;DR

This paper proposes two-scale loss functions for classification tasks in machine learning, enhancing focus on misclassified objects and improving performance across various neural network architectures and datasets.

Contribution

Introduction of a generic two-scale loss function applicable to diverse machine learning models, improving classification accuracy by emphasizing poorly classified objects.

Findings

Improved performance over cross-entropy loss on MNIST, CIFAR10, and CIFAR100 datasets.

Applicable to various architectures including deep neural networks and support vector machines.

Enhances focus on misclassified objects during training.

Abstract

We introduce two-scale loss functions for use in various gradient descent algorithms applied to classification problems via deep neural networks. This new method is generic in the sense that it can be applied to a wide range of machine learning architectures, from deep neural networks to support vector machines for example. These two-scale loss functions allow to focus the training onto objects in the training set which are not well classified. This leads to an increase in several measures of performance for appropriately-defined two-scale loss functions with respect to the more classical cross-entropy when tested on traditional deep neural networks on the MNIST, CIFAR10, and CIFAR100 data-sets.