Joint Active Learning with Feature Selection via CUR Matrix Decomposition

TL;DR

This paper introduces a novel unsupervised framework that jointly performs sample and feature selection using CUR matrix decomposition, effectively handling high-dimensional data without iterative labeling.

Contribution

It proposes a one-shot joint active learning and feature selection method based on CUR decomposition, suitable for scenarios with limited or no labeled data.

Findings

Outperforms state-of-the-art methods on public datasets

Efficient one-shot approach without iterative labeling

Proven global convergence of the optimization algorithm

Abstract

This paper presents an unsupervised learning approach for simultaneous sample and feature selection, which is in contrast to existing works which mainly tackle these two problems separately. In fact the two tasks are often interleaved with each other: noisy and high-dimensional features will bring adverse effect on sample selection, while informative or representative samples will be beneficial to feature selection. Specifically, we propose a framework to jointly conduct active learning and feature selection based on the CUR matrix decomposition. From the data reconstruction perspective, both the selected samples and features can best approximate the original dataset respectively, such that the selected samples characterized by the features are highly representative. In particular, our method runs in one-shot without the procedure of iterative sample selection for progressive labeling. Thus, our model is especially suitable when there are few labeled samples or even in the absence of supervision, which is a particular challenge for existing methods. As the joint learning problem is NP-hard, the proposed formulation involves a convex but non-smooth optimization problem. We solve it efficiently by an iterative algorithm, and prove its global convergence. Experimental results on publicly available datasets corroborate the efficacy of our method compared with the state-of-the-art.