Towards Practicable Sequential Shift Detectors

TL;DR

This paper discusses the importance of practical sequential shift detectors for machine learning models, identifies key desiderata for deployment, reviews existing solutions, and suggests future research directions.

Contribution

It highlights overlooked practical requirements for shift detectors, reviews related work, and proposes impactful future research directions.

Findings

Identified three key desiderata for practical shift detectors.

Reviewed existing works relevant to these desiderata.

Recommended impactful directions for future research.

Abstract

There is a growing awareness of the harmful effects of distribution shift on the performance of deployed machine learning models. Consequently, there is a growing interest in detecting these shifts before associated costs have time to accumulate. However, desiderata of crucial importance to the practicable deployment of sequential shift detectors are typically overlooked by existing works, precluding their widespread adoption. We identify three such desiderata, highlight existing works relevant to their satisfaction, and recommend impactful directions for future research.