# Effects of the optimisation of the margin distribution on generalisation in deep architectures

## TL;DR

This paper proposes a new loss function called Halfway loss that minimizes margin variance instead of maximizing margin, leading to improved generalisation in deep neural networks.

## Contribution

It introduces the Halfway loss function that focuses on reducing margin variance, a novel approach for enhancing deep learning model generalisation.

## Key findings

- Halfway loss outperforms Softmax Cross-Entropy on multiple datasets.
- Minimising margin variance improves deep model generalisation.
- The approach offers a new perspective on margin principles in deep learning.

## Abstract

Despite being so vital to success of Support Vector Machines, the principle of separating margin maximisation is not used in deep learning. We show that minimisation of margin variance and not maximisation of the margin is more suitable for improving generalisation in deep architectures. We propose the Halfway loss function that minimises the Normalised Margin Variance (NMV) at the output of a deep learning models and evaluate its performance against the Softmax Cross-Entropy loss on the MNIST, smallNORB and CIFAR-10 datasets.