An analysis of the least median of squares regression problem

TL;DR

This paper analyzes the optimization landscape of least median of squares regression, providing exact counts of local minima and proposing three algorithms for solving the problem.

Contribution

It offers a novel mathematical characterization of the local minima in least median of squares regression and introduces three algorithms based on this analysis.

Findings

Exact number of local minima derived as a combinatorial formula

Representation of the median of absolute residuals problem

Outline of three algorithms for regression analysis

Abstract

The optimization problem that arises out of the least median of squared residuals method in linear regression is analyzed. To simplify the analysis, the problem is replaced by an equivalent one of minimizing the median of absolute residuals. A useful representation of the last problem is given to examine properties of the objective function and estimate the number of its local minima. It is shown that the exact number of local minima is equal to $ {p+\lfloor (n-1)/2 \rfloor \choose{p}} $, where $ p $ is the dimension of the regression model and $ n $ is the number of observations. As applications of the results, three algorithms are also outlined.