Space Alternating Penalized Kullback Proximal Point Algorithms for Maximizing Likelihood with Nondifferentiable Penalty

TL;DR

This paper introduces a novel space alternating penalized Kullback proximal algorithm for maximizing likelihood with nonsmooth penalties, ensuring convergence to stationary points even on boundaries, with applications in mixture models and image reconstruction.

Contribution

It proposes a new class of algorithms extending the EM method to nonsmooth penalties, with component-wise implementation for complex models.

Findings

Cluster points are stationary even on boundary.

Algorithm successfully applied to mixture model selection.

Effective in sparse image reconstruction.

Abstract

The EM algorithm is a widely used methodology for penalized likelihood estimation. Provable monotonicity and convergence are the hallmarks of the EM algorithm and these properties are well established for smooth likelihood and smooth penalty functions. However, many relaxed versions of variable selection penalties are not smooth. The goal of this paper is to introduce a new class of Space Alternating Penalized Kullback Proximal extensions of the EM algorithm for nonsmooth likelihood inference. We show that the cluster points of the new method are stationary points even when on the boundary of the parameter set. Special attention has been paid to the construction of component-wise version of the method in order to ease the implementation for complicated models. Illustration for the problems of model selection for finite mixtures of regression and to sparse image reconstruction is presented.