General oracle inequalities for a penalized log-likelihood criterion based on non-stationary data

TL;DR

This paper establishes oracle inequalities for penalized log-likelihood methods applicable to non-stationary, dependent data using a martingale approach, extending theoretical guarantees across various complex models.

Contribution

It introduces a martingale-based framework for oracle inequalities that applies to non-stationary, dependent data, broadening the scope of existing theoretical results.

Findings

Validates assumptions for multiple data contexts

Achieves comparable or improved bounds in dependent data scenarios

Extends oracle inequalities to non-stationary, dependent data

Abstract

We prove oracle inequalities for a penalized log-likelihood criterion that hold even if the data are not independent and not stationary, based on a martingale approach. The assumptions are checked for various contexts: density estimation with independent and identically distributed (i.i.d) data, hidden Markov models, spiking neural networks, adversarial bandits. In each case, we compare our results to the literature, showing that, although we lose some logarithmic factors in the most classical case (i.i.d.), these results are comparable or more general than the existing results in the more dependent cases.