Max-Mahalanobis Linear Discriminant Analysis Networks

TL;DR

This paper introduces a novel neural network classifier, MM-LDA, that enhances robustness to adversarial attacks by explicitly mapping data to a Max-Mahalanobis distribution before classification.

Contribution

It proposes the MM-LDA network that maps data to a Max-Mahalanobis distribution in feature space, improving adversarial robustness and classification performance.

Findings

MM-LDA networks are more robust to adversarial attacks.

MM-LDA achieves better class-biased classification performance.

Theoretically, LDA is optimal under Max-Mahalanobis distribution.

Abstract

A deep neural network (DNN) consists of a nonlinear transformation from an input to a feature representation, followed by a common softmax linear classifier. Though many efforts have been devoted to designing a proper architecture for nonlinear transformation, little investigation has been done on the classifier part. In this paper, we show that a properly designed classifier can improve robustness to adversarial attacks and lead to better prediction results. Specifically, we define a Max-Mahalanobis distribution (MMD) and theoretically show that if the input distributes as a MMD, the linear discriminant analysis (LDA) classifier will have the best robustness to adversarial examples. We further propose a novel Max-Mahalanobis linear discriminant analysis (MM-LDA) network, which explicitly maps a complicated data distribution in the input space to a MMD in the latent feature space and then applies LDA to make predictions. Our results demonstrate that the MM-LDA networks are significantly more robust to adversarial attacks, and have better performance in class-biased classification.