Using Time-Series Privileged Information for Provably Efficient Learning of Prediction Models

TL;DR

This paper introduces a method for improving prediction models by leveraging privileged time-series data available only during training, demonstrating theoretical and empirical benefits in sample efficiency especially with limited data.

Contribution

The paper proposes an algorithm for using privileged time-series information in supervised learning, with theoretical guarantees for non-stationary Gaussian-linear systems and empirical validation on real datasets.

Findings

Privileged information improves sample efficiency in prediction models.

The approach outperforms classical methods when data is scarce.

The estimator relates to a distillation approach both theoretically and empirically.

Abstract

We study prediction of future outcomes with supervised models that use privileged information during learning. The privileged information comprises samples of time series observed between the baseline time of prediction and the future outcome; this information is only available at training time which differs from the traditional supervised learning. Our question is when using this privileged data leads to more sample-efficient learning of models that use only baseline data for predictions at test time. We give an algorithm for this setting and prove that when the time series are drawn from a non-stationary Gaussian-linear dynamical system of fixed horizon, learning with privileged information is more efficient than learning without it. On synthetic data, we test the limits of our algorithm and theory, both when our assumptions hold and when they are violated. On three diverse real-world datasets, we show that our approach is generally preferable to classical learning, particularly when data is scarce. Finally, we relate our estimator to a distillation approach both theoretically and empirically.