Asymptotic and finite-sample distributions of one- and two-sample empirical relative entropy, with application to change-point detection

TL;DR

This paper analyzes the distribution of empirical relative entropy for change-point detection, providing theoretical approximations and demonstrating its effectiveness on real datasets compared to classical methods.

Contribution

It introduces new distributional results and a change-point detection method based on empirical relative entropy, with extensive theoretical and empirical validation.

Findings

Derived finite-sample concentration inequalities.

Established asymptotic and Berry-Esseen bounds.

Demonstrated improved detection performance on real data.

Abstract

Relative entropy, as a divergence metric between two distributions, can be used for offline change-point detection and extends classical methods that mainly rely on moment-based discrepancies. To build a statistical test suitable for this context, we study the distribution of empirical relative entropy and derive several types of approximations: concentration inequalities for finite samples, asymptotic distributions, and Berry-Esseen bounds in a pre-asymptotic regime. For the latter, we introduce a new approach to obtain Berry-Esseen inequalities for nonlinear functions of sum statistics under some convexity assumptions. Our theoretical contributions cover both one- and two-sample empirical relative entropies. We then detail a change-point detection procedure built on relative entropy and compare it, through extensive simulations, with classical methods based on moments or on information criteria. Finally, we illustrate its practical relevance on two real datasets involving temperature series and volatility of stock indices.