Learning from Snapshots of Discrete and Continuous Data Streams

TL;DR

This paper explores theoretical frameworks for learning from data streams, demonstrating how adaptive algorithms can effectively learn certain classes of functions while others remain unlearnable, with implications for online learning scenarios.

Contribution

It introduces two frameworks for learning from data streams, providing algorithms and theoretical insights into what can be learned under different conditions.

Findings

Uniform sampling can learn concept classes with finite Littlestone dimension.

Non-trivial concept classes are unlearnable in the blind-prediction setting.

Adaptive algorithms are essential for learning pattern classes in data streams.

Abstract

Imagine a smart camera trap selectively clicking pictures to understand animal movement patterns within a particular habitat. These "snapshots", or pieces of data captured from a data stream at adaptively chosen times, provide a glimpse of different animal movements unfolding through time. Learning a continuous-time process through snapshots, such as smart camera traps, is a central theme governing a wide array of online learning situations. In this paper, we adopt a learning-theoretic perspective in understanding the fundamental nature of learning different classes of functions from both discrete data streams and continuous data streams. In our first framework, the \textit{update-and-deploy} setting, a learning algorithm discretely queries from a process to update a predictor designed to make predictions given as input the data stream. We construct a uniform sampling algorithm that can learn with bounded error any concept class with finite Littlestone dimension. Our second framework, known as the \emph{blind-prediction} setting, consists of a learning algorithm generating predictions independently of observing the process, only engaging with the process when it chooses to make queries. Interestingly, we show a stark contrast in learnability where non-trivial concept classes are unlearnable. However, we show that adaptive learning algorithms are necessary to learn sets of time-dependent and data-dependent functions, called pattern classes, in either framework. Finally, we develop a theory of pattern classes under discrete data streams for the blind-prediction setting.