Online Feature Screening for Data Streams with Concept Drift

TL;DR

This paper introduces online feature screening methods capable of handling high-dimensional, streaming data with sparsity and concept drift, demonstrating faster processing and improved feature detection in classification tasks.

Contribution

The proposed online screening methods are novel in their ability to handle streaming data with concept drift, offering comparable feature importance to offline methods with enhanced efficiency.

Findings

Methods match offline feature importance with faster speed.

Online screening with model adaptation detects true features better.

Advantages include reduced computation, storage, and improved accuracy.

Abstract

Screening feature selection methods are often used as a preprocessing step for reducing the number of variables before training step. Traditional screening methods only focus on dealing with complete high dimensional datasets. Modern datasets not only have higher dimension and larger sample size, but also have properties such as streaming input, sparsity and concept drift. Therefore a considerable number of online feature selection methods were introduced to handle these kind of problems in recent years. Online screening methods are one of the categories of online feature selection methods. The methods that we proposed in this research are capable of handling all three situations mentioned above. Our research study focuses on classification datasets. Our experiments show proposed methods can generate the same feature importance as their offline version with faster speed and less storage consumption. Furthermore, the results show that online screening methods with integrated model adaptation have a higher true feature detection rate than without model adaptation on data streams with the concept drift property. Among the two large real datasets that potentially have the concept drift property, online screening methods with model adaptation show advantages in either saving computing time and space, reducing model complexity, or improving prediction accuracy.