# $\mathcal{G}$-softmax: Improving Intra-class Compactness and Inter-class Separability of Features

## TL;DR

This paper introduces $\\mathcal{G}$-softmax, a Gaussian-based softmax function that enhances intra-class compactness and inter-class separability of features, leading to improved classification performance across multiple datasets.

## Contribution

The paper proposes a simple, effective Gaussian-based softmax function that replaces the standard softmax to improve feature discriminability in convolutional networks.

## Key findings

- $\\mathcal{G}$-softmax improves state-of-the-art accuracy on CIFAR-10, CIFAR-100, Tiny ImageNet, MS COCO, and NUS-WIDE.
- Enhanced intra-class compactness and inter-class separability correlate linearly with higher average precision.
- The proposed method is easy to implement and integrates seamlessly into existing models.

## Abstract

Intra-class compactness and inter-class separability are crucial indicators to measure the effectiveness of a model to produce discriminative features, where intra-class compactness indicates how close the features with the same label are to each other and inter-class separability indicates how far away the features with different labels are. In this work, we investigate intra-class compactness and inter-class separability of features learned by convolutional networks and propose a Gaussian-based softmax ($\mathcal{G}$-softmax) function that can effectively improve intra-class compactness and inter-class separability. The proposed function is simple to implement and can easily replace the softmax function. We evaluate the proposed $\mathcal{G}$-softmax function on classification datasets (i.e., CIFAR-10, CIFAR-100, and Tiny ImageNet) and on multi-label classification datasets (i.e., MS COCO and NUS-WIDE). The experimental results show that the proposed $\mathcal{G}$-softmax function improves the state-of-the-art models across all evaluated datasets. In addition, analysis of the intra-class compactness and inter-class separability demonstrates the advantages of the proposed function over the softmax function, which is consistent with the performance improvement. More importantly, we observe that high intra-class compactness and inter-class separability are linearly correlated to average precision on MS COCO and NUS-WIDE. This implies that improvement of intra-class compactness and inter-class separability would lead to improvement of average precision.