Learning with Changing Features

TL;DR

This paper introduces a method to detect when features start to influence an output variable in dynamic settings, with applications in retail and manufacturing, providing theoretical guarantees and real-world case studies.

Contribution

It presents the first formal approach for change point detection in a distribution-independent, supervised learning context, with provable performance and practical efficiency.

Findings

Successfully identified change points in retail data linked to news events.

Detected feature relevance shifts in manufacturing plant data.

Method is efficient and applicable to multiple change points.

Abstract

In this paper we study the setting where features are added or change interpretation over time, which has applications in multiple domains such as retail, manufacturing, finance. In particular, we propose an approach to provably determine the time instant from which the new/changed features start becoming relevant with respect to an output variable in an agnostic (supervised) learning setting. We also suggest an efficient version of our approach which has the same asymptotic performance. Moreover, our theory also applies when we have more than one such change point. Independent post analysis of a change point identified by our method for a large retailer revealed that it corresponded in time with certain unflattering news stories about a brand that resulted in the change in customer behavior. We also applied our method to data from an advanced manufacturing plant identifying the time instant from which downstream features became relevant. To the best of our knowledge this is the first work that formally studies change point detection in a distribution independent agnostic setting, where the change point is based on the changing relationship between input and output.