Introducing One Sided Margin Loss for Solving Classification Problems in Deep Networks

TL;DR

This paper proposes the One-Sided Margin (OSM) loss function for deep neural network classification, demonstrating faster training and higher accuracy across various datasets and network sizes compared to traditional loss functions.

Contribution

The introduction of the OSM loss function, explicitly controlling the margin, offers a more effective and efficient alternative to hinge and cross-entropy losses in deep learning classification tasks.

Findings

OSM achieves state-of-the-art accuracy on CIFAR10, CIFAR100, Flowers, and Stanford Cars datasets.

OSM leads to faster training speeds than cross-entropy and hinge losses.

OSM outperforms traditional loss functions across small and large neural networks.

Abstract

This paper introduces a new loss function, OSM (One-Sided Margin), to solve maximum-margin classification problems effectively. Unlike the hinge loss, in OSM the margin is explicitly determined with corresponding hyperparameters and then the classification problem is solved. In experiments, we observe that using OSM loss leads to faster training speeds and better accuracies than binary and categorical cross-entropy in several commonly used deep models for classification and optical character recognition problems. OSM has consistently shown better classification accuracies over cross-entropy and hinge losses for small to large neural networks. it has also led to a more efficient training procedure. We achieved state-of-the-art accuracies for small networks on several benchmark datasets of CIFAR10(98.82\%), CIFAR100(91.56\%), Flowers(98.04\%), Stanford Cars(93.91\%) with considerable improvements over other loss functions. Moreover, the accuracies are rather better than cross-entropy and hinge loss for large networks. Therefore, we strongly believe that OSM is a powerful alternative to hinge and cross-entropy losses to train deep neural networks on classification tasks.