Regularized Modal Regression on Markov-dependent Observations: A Theoretical Assessment

TL;DR

This paper investigates the theoretical properties of regularized modal regression when observations are Markov-dependent, revealing how dependence affects generalization error and providing explicit learning rates.

Contribution

It extends the theoretical analysis of modal regression to Markov-dependent data, deriving bounds and learning rates that account for dependence structure.

Findings

Markov dependence influences generalization error via spectral gap.

Established upper bounds for RMR estimator under dependence.

Provided explicit learning rates for Markov-dependent modal regression.

Abstract

Modal regression, a widely used regression protocol, has been extensively investigated in statistical and machine learning communities due to its robustness to outliers and heavy-tailed noises. Understanding modal regression's theoretical behavior can be fundamental in learning theory. Despite significant progress in characterizing its statistical property, the majority of the results are based on the assumption that samples are independent and identical distributed (i.i.d.), which is too restrictive for real-world applications. This paper concerns the statistical property of regularized modal regression (RMR) within an important dependence structure - Markov dependent. Specifically, we establish the upper bound for RMR estimator under moderate conditions and give an explicit learning rate. Our results show that the Markov dependence impacts on the generalization error in the way that sample size would be discounted by a multiplicative factor depending on the spectral gap of underlying Markov chain. This result shed a new light on characterizing the theoretical underpinning for robust regression.