Holonomic extended least angle regression

TL;DR

This paper introduces HELARS, a novel algorithm combining holonomic gradient methods with extended LARS to efficiently perform model fitting in complex distributions where normalizing constants satisfy holonomic systems.

Contribution

The paper presents the development and implementation of HELARS, extending LARS with holonomic gradient techniques to handle complex normalizing constants in generalized linear models.

Findings

Successfully implemented in R

Effective on real and simulated datasets

Handles complex normalizing constants satisfying holonomic systems

Abstract

One of the main problems studied in statistics is the fitting of models. Ideally, we would like to explain a large dataset with as few parameters as possible. There have been numerous attempts at automatizing this process. Most notably, the Least Angle Regression algorithm, or LARS, is a computationally efficient algorithm that ranks the covariates of a linear model. The algorithm is further extended to a class of distributions in the generalized linear model by using properties of the manifold of exponential families as dually flat manifolds. However this extension assumes that the normalizing constant of the joint distribution of observations is easy to compute. This is often not the case, for example the normalizing constant may contain a complicated integral. We circumvent this issue if the normalizing constant satisfies a holonomic system, a system of linear partial differential equations with a finite-dimensional space of solutions. In this paper we present a modification of the holonomic gradient method and add it to the extended LARS algorithm. We call this the holonomic extended least angle regression algorithm, or HELARS. The algorithm was implemented using the statistical software R, and was tested with real and simulated datasets.