Learning with Feature Evolvable Streams

TL;DR

This paper introduces a new learning paradigm for streaming data where features can evolve over time, proposing methods to recover old features and improve learning performance through ensemble techniques.

Contribution

It proposes the concept of Feature Evolvable Streaming Learning and develops ensemble methods to leverage recovered old features for enhanced model performance.

Findings

Ensemble methods improve prediction accuracy with evolving features.

Theoretical guarantees show performance benefits of using recovered features.

Experimental results validate the effectiveness on synthetic and real datasets.

Abstract

Learning with streaming data has attracted much attention during the past few years. Though most studies consider data stream with fixed features, in real practice the features may be evolvable. For example, features of data gathered by limited-lifespan sensors will change when these sensors are substituted by new ones. In this paper, we propose a novel learning paradigm: \emph{Feature Evolvable Streaming Learning} where old features would vanish and new features would occur. Rather than relying on only the current features, we attempt to recover the vanished features and exploit it to improve performance. Specifically, we learn two models from the recovered features and the current features, respectively. To benefit from the recovered features, we develop two ensemble methods. In the first method, we combine the predictions from two models and theoretically show that with the assistance of old features, the performance on new features can be improved. In the second approach, we dynamically select the best single prediction and establish a better performance guarantee when the best model switches. Experiments on both synthetic and real data validate the effectiveness of our proposal.