Loss Function Search for Face Recognition

TL;DR

This paper introduces a novel AutoML-based method for automatically searching for optimal loss functions in face recognition, focusing on reducing softmax probability to improve feature discrimination.

Contribution

It proposes a unified formulation and a reward-guided search method for margin-based softmax losses, outperforming existing methods in face recognition benchmarks.

Findings

Outperforms state-of-the-art loss functions in face recognition benchmarks

Effectively reduces softmax probability to enhance feature discrimination

Provides a unified framework for margin-based softmax loss search

Abstract

In face recognition, designing margin-based (e.g., angular, additive, additive angular margins) softmax loss functions plays an important role in learning discriminative features. However, these hand-crafted heuristic methods are sub-optimal because they require much effort to explore the large design space. Recently, an AutoML for loss function search method AM-LFS has been derived, which leverages reinforcement learning to search loss functions during the training process. But its search space is complex and unstable that hindering its superiority. In this paper, we first analyze that the key to enhance the feature discrimination is actually \textbf{how to reduce the softmax probability}. We then design a unified formulation for the current margin-based softmax losses. Accordingly, we define a novel search space and develop a reward-guided search method to automatically obtain the best candidate. Experimental results on a variety of face recognition benchmarks have demonstrated the effectiveness of our method over the state-of-the-art alternatives.