Spatial aggregation of local likelihood estimates with applications to classification

TL;DR

This paper introduces a spatially adaptive likelihood estimation method that aggregates local estimates to achieve near-optimal risk, applicable to various models and demonstrated through classification tasks.

Contribution

The paper proposes a novel aggregation technique for local likelihood estimates that adaptively minimizes risk and includes a new parameter selection approach with theoretical optimality guarantees.

Findings

The aggregated estimate achieves risk close to the best among all local estimates.

The method performs well in classification problems, both in simulations and real data.

Theoretical results establish the near-optimality of the aggregation procedure.

Abstract

This paper presents a new method for spatially adaptive local (constant) likelihood estimation which applies to a broad class of nonparametric models, including the Gaussian, Poisson and binary response models. The main idea of the method is, given a sequence of local likelihood estimates (``weak'' estimates), to construct a new aggregated estimate whose pointwise risk is of order of the smallest risk among all ``weak'' estimates. We also propose a new approach toward selecting the parameters of the procedure by providing the prescribed behavior of the resulting estimate in the simple parametric situation. We establish a number of important theoretical results concerning the optimality of the aggregated estimate. In particular, our ``oracle'' result claims that its risk is, up to some logarithmic multiplier, equal to the smallest risk for the given family of estimates. The performance of the procedure is illustrated by application to the classification problem. A numerical study demonstrates its reasonable performance in simulated and real-life examples.