Quantized Fisher Discriminant Analysis

TL;DR

This paper introduces Quantized Fisher Discriminant Analysis (QFDA), a novel subspace learning method that maintains classification accuracy on compressed images, combining machine learning and information theory principles.

Contribution

QFDA is the first method to optimize discriminant analysis directly on quantized images in the DCT domain, bridging a gap between machine learning and information theory.

Findings

QFDA achieves comparable classification accuracy on compressed images.

Experiments on AT&T and Fashion MNIST datasets validate effectiveness.

Proposed cost function aligns with rate-distortion optimization.

Abstract

This paper proposes a new subspace learning method, named Quantized Fisher Discriminant Analysis (QFDA), which makes use of both machine learning and information theory. There is a lack of literature for combination of machine learning and information theory and this paper tries to tackle this gap. QFDA finds a subspace which discriminates the uniformly quantized images in the Discrete Cosine Transform (DCT) domain at least as well as discrimination of non-quantized images by Fisher Discriminant Analysis (FDA) while the images have been compressed. This helps the user to throw away the original images and keep the compressed images instead without noticeable loss of classification accuracy. We propose a cost function whose minimization can be interpreted as rate-distortion optimization in information theory. We also propose quantized Fisherfaces for facial analysis in QFDA. Our experiments on AT&T face dataset and Fashion MNIST dataset show the effectiveness of this subspace learning method.