Diverse Online Feature Selection

TL;DR

This paper introduces a novel online feature selection method using Determinantal Point Processes to promote diversity, combining sampling, local, and global criteria, and demonstrates improved performance over existing methods.

Contribution

It presents a new diverse online feature selection framework based on DPP that integrates multiple criteria for better feature subset selection.

Findings

Outperforms or matches state-of-the-art online feature selection methods.

Produces more compact and diverse feature subsets.

Effective in both supervised and unsupervised settings.

Abstract

Online feature selection has been an active research area in recent years. We propose a novel diverse online feature selection method based on Determinantal Point Processes (DPP). Our model aims to provide diverse features which can be composed in either a supervised or unsupervised framework. The framework aims to promote diversity based on the kernel produced on a feature level, through at most three stages: feature sampling, local criteria and global criteria for feature selection. In the feature sampling, we sample incoming stream of features using conditional DPP. The local criteria is used to assess and select streamed features (i.e. only when they arrive), we use unsupervised scale invariant methods to remove redundant features and optionally supervised methods to introduce label information to assess relevant features. Lastly, the global criteria uses regularization methods to select a global optimal subset of features. This three stage procedure continues until there are no more features arriving or some predefined stopping condition is met. We demonstrate based on experiments conducted on that this approach yields better compactness, is comparable and in some instances outperforms other state-of-the-art online feature selection methods.