Kernel Reconstruction ICA for Sparse Representation

TL;DR

This paper introduces kernel and supervised variants of ICA with reconstruction constraints to learn nonlinear, sparse, and discriminative representations for improved image classification.

Contribution

It proposes kRICA and d-kRICA models that incorporate nonlinear kernels and class information into ICA for better sparse representation and classification performance.

Findings

kRICA effectively captures nonlinear features.

d-kRICA learns class-structured sparse representations.

Experimental results show improved image classification accuracy.

Abstract

Independent Component Analysis (ICA) is an effective unsupervised tool to learn statistically independent representation. However, ICA is not only sensitive to whitening but also difficult to learn an over-complete basis. Consequently, ICA with soft Reconstruction cost(RICA) was presented to learn sparse representations with over-complete basis even on unwhitened data. Whereas RICA is infeasible to represent the data with nonlinear structure due to its intrinsic linearity. In addition, RICA is essentially an unsupervised method and can not utilize the class information. In this paper, we propose a kernel ICA model with reconstruction constraint (kRICA) to capture the nonlinear features. To bring in the class information, we further extend the unsupervised kRICA to a supervised one by introducing a discrimination constraint, namely d-kRICA. This constraint leads to learn a structured basis consisted of basis vectors from different basis subsets corresponding to different class labels. Then each subset will sparsely represent well for its own class but not for the others. Furthermore, data samples belonging to the same class will have similar representations, and thereby the learned sparse representations can take more discriminative power. Experimental results validate the effectiveness of kRICA and d-kRICA for image classification.