Two convergence results for an alternation maximization procedure

TL;DR

This paper establishes finite sample deviation bounds and nearly linear convergence for an alternating maximization procedure, extending results from profile M-estimators to iterative maximization algorithms.

Contribution

It derives two convergence results for an alternating maximization procedure, including deviation bounds and conditions for near-linear convergence.

Findings

Sequence attains deviation properties similar to profile M-estimators.

Finite sample Wilks and Fisher theorems are applicable.

Nearly linear convergence under smoothness and good initialization.

Abstract

Andresen and Spokoiny's (2013) ``critical dimension in semiparametric estimation`` provide a technique for the finite sample analysis of profile M-estimators. This paper uses very similar ideas to derive two convergence results for the alternating procedure to approximate the maximizer of random functionals such as the realized log likelihood in MLE estimation. We manage to show that the sequence attains the same deviation properties as shown for the profile M-estimator in Andresen and Spokoiny (2013), i.e. a finite sample Wilks and Fisher theorem. Further under slightly stronger smoothness constraints on the random functional we can show nearly linear convergence to the global maximizer if the starting point for the procedure is well chosen.