Sufficient Component Analysis for Supervised Dimension Reduction

TL;DR

This paper introduces Sufficient Component Analysis (SCA), a new distribution-free method for supervised dimension reduction that is computationally efficient and effective for real-world image and audio data.

Contribution

SCA is a novel, distribution-free SDR method that improves computational efficiency using dependence estimation and maximization with LSMI and Epanechnikov kernel.

Findings

SCA outperforms existing methods in large-scale experiments

SCA is computationally more efficient than previous SDR techniques

SCA effectively reduces dimensions in image and audio classification tasks

Abstract

The purpose of sufficient dimension reduction (SDR) is to find the low-dimensional subspace of input features that is sufficient for predicting output values. In this paper, we propose a novel distribution-free SDR method called sufficient component analysis (SCA), which is computationally more efficient than existing methods. In our method, a solution is computed by iteratively performing dependence estimation and maximization: Dependence estimation is analytically carried out by recently-proposed least-squares mutual information (LSMI), and dependence maximization is also analytically carried out by utilizing the Epanechnikov kernel. Through large-scale experiments on real-world image classification and audio tagging problems, the proposed method is shown to compare favorably with existing dimension reduction approaches.