Lazy FSCA for Unsupervised Variable Selection

TL;DR

This paper introduces a 'lazy' implementation of the FSCA algorithm for unsupervised variable selection, significantly reducing computation time while maintaining similar performance levels, validated through extensive experiments.

Contribution

It proposes L-FSCA, a faster variant of FSCA that retains effectiveness despite the non-submodular criterion, and demonstrates its efficiency through comprehensive case studies.

Findings

L-FSCA reduces computation time by up to 94%.

L-FSCA achieves performance comparable to FSCA.

Experimental results validate L-FSCA's efficiency and effectiveness.

Abstract

Various unsupervised greedy selection methods have been proposed as computationally tractable approximations to the NP-hard subset selection problem. These methods rely on sequentially selecting the variables that best improve performance with respect to a selection criterion. Theoretical results exist that provide performance bounds and enable "lazy greedy" efficient implementations for selection criteria that satisfy a diminishing returns property known as submodularity. This has motivated the development of variable selection algorithms based on mutual information and frame potential. Recently, the authors introduced Forward Selection Component Analysis (FSCA) which uses variance explained as its selection criterion. While this criterion is not submodular, FSCA has been shown to be highly effective for applications such as measurement plan optimisation. In this paper a "lazy" implementation of the FSCA algorithm (L-FSCA) is proposed, which, although not equivalent to FSCA due to the absence of submodularity, has the potential to yield comparable performance while being up to an order of magnitude faster to compute. The efficacy of L-FSCA is demonstrated by performing a systematic comparison with FSCA and five other unsupervised variable selection methods from the literature using simulated and real-world case studies. Experimental results confirm that L-FSCA yields almost identical performance to FSCA while reducing computation time by between 22% and 94% for the case studies considered.