Imbalance Robust Softmax for Deep Embeeding Learning

TL;DR

This paper introduces IR-Softmax, a unified softmax-based framework that addresses data imbalance and open-set challenges in deep embedding learning for face recognition and re-identification, improving performance across multiple datasets.

Contribution

The paper proposes IR-Softmax, a novel approach that aligns class weights with class centers to mitigate data imbalance effects and enhances existing softmax variants for open-set recognition.

Findings

IR-Softmax outperforms state-of-the-art methods on face recognition datasets.

Aligning weights with class centers reduces performance degradation due to data imbalance.

The framework generalizes to various softmax variants, improving their robustness.

Abstract

Deep embedding learning is expected to learn a metric space in which features have smaller maximal intra-class distance than minimal inter-class distance. In recent years, one research focus is to solve the open-set problem by discriminative deep embedding learning in the field of face recognition (FR) and person re-identification (re-ID). Apart from open-set problem, we find that imbalanced training data is another main factor causing the performance degradation of FR and re-ID, and data imbalance widely exists in the real applications. However, very little research explores why and how data imbalance influences the performance of FR and re-ID with softmax or its variants. In this work, we deeply investigate data imbalance in the perspective of neural network optimisation and feature distribution about softmax. We find one main reason of performance degradation caused by data imbalance is that the weights (from the penultimate fully-connected layer) are far from their class centers in feature space. Based on this investigation, we propose a unified framework, Imbalance-Robust Softmax (IR-Softmax), which can simultaneously solve the open-set problem and reduce the influence of data imbalance. IR-Softmax can generalise to any softmax and its variants (which are discriminative for open-set problem) by directly setting the weights as their class centers, naturally solving the data imbalance problem. In this work, we explicitly re-formulate two discriminative softmax (A-Softmax and AM-Softmax) under the framework of IR-Softmax. We conduct extensive experiments on FR databases (LFW, MegaFace) and re-ID database (Market-1501, Duke), and IR-Softmax outperforms many state-of-the-art methods.