Sparse Kernel Canonical Correlation Analysis via $\ell_1$-regularization

TL;DR

This paper introduces SKCCA, a sparse kernel CCA algorithm that employs $ ext{l}_1$-regularization to produce sparse solutions, improve interpretability, and reduce overfitting in nonlinear relation analysis.

Contribution

The paper proposes a novel SKCCA algorithm that integrates $ ext{l}_1$-regularization into kernel CCA, enabling sparse solutions and addressing overfitting issues.

Findings

Effective in computing sparse dual transformations

Reduces overfitting in kernel CCA

Performs well in experiments

Abstract

Canonical correlation analysis (CCA) is a multivariate statistical technique for finding the linear relationship between two sets of variables. The kernel generalization of CCA named kernel CCA has been proposed to find nonlinear relations between datasets. Despite their wide usage, they have one common limitation that is the lack of sparsity in their solution. In this paper, we consider sparse kernel CCA and propose a novel sparse kernel CCA algorithm (SKCCA). Our algorithm is based on a relationship between kernel CCA and least squares. Sparsity of the dual transformations is introduced by penalizing the $\ell_{1}$-norm of dual vectors. Experiments demonstrate that our algorithm not only performs well in computing sparse dual transformations but also can alleviate the over-fitting problem of kernel CCA.