# Variational Analysis of Constrained M-Estimators

## TL;DR

This paper introduces a comprehensive framework for analyzing nonparametric M-estimators, ensuring their existence, computability, and strong consistency even with complex function classes and limited data.

## Contribution

It provides a unified approach to establish existence, consistency, and computation of M-estimators for rich, complex function classes using the Attouch-Wets metric.

## Key findings

- Framework guarantees strong consistency of M-estimators.
- Enables systematic treatment of various estimators like mode and level-sets.
- Allows computation via approximation with parametric classes.

## Abstract

We propose a unified framework for establishing existence of nonparametric M-estimators, computing the corresponding estimates, and proving their strong consistency when the class of functions is exceptionally rich. In particular, the framework addresses situations where the class of functions is complex involving information and assumptions about shape, pointwise bounds, location of modes, height at modes, location of level-sets, values of moments, size of subgradients, continuity, distance to a "prior" function, multivariate total positivity, and any combination of the above. The class might be engineered to perform well in a specific setting even in the presence of little data. The framework views the class of functions as a subset of a particular metric space of upper semicontinuous functions under the Attouch-Wets distance. In addition to allowing a systematic treatment of numerous M-estimators, the framework yields consistency of plug-in estimators of modes of densities, maximizers of regression functions, level-sets of classifiers, and related quantities, and also enables computation by means of approximating parametric classes. We establish consistency through a one-sided law of large numbers, here extended to sieves, that relaxes assumptions of uniform laws, while ensuring global approximations even under model misspecification.