Laplace Approximations for Mixed-Effects and Gaussian Process Quantile Regression

TL;DR

This paper develops a novel Laplace approximation framework for quantile regression with mixed-effects and Gaussian process models, overcoming previous limitations by using Fisher information and population curvature.

Contribution

It introduces practical curvature estimators, including the triangular kernel curvature, and establishes their asymptotic validity for scalable, accurate inference in non-smooth models.

Findings

Methods are scalable and numerically stable.

Achieve accuracy comparable or better than MCMC and variational methods.

Lower computational costs than traditional approaches.

Abstract

Laplace approximations are a standard tool for computationally efficient inference in latent Gaussian models, but they fail for quantile regression with the asymmetric Laplace likelihood because the observed Hessian vanishes almost everywhere. We show that this obstacle can be overcome without smoothing the likelihood: the relevant local curvature is given not by the observed Hessian, but by the Fisher information when the model is correctly specified and by the population curvature of the expected loss under misspecification. On this basis, we develop a Laplace approximation framework for quantile regression with mixed-effects and Gaussian process models. We propose practical curvature estimators, including the triangular kernel curvature (TKC) estimator, that yield approximations for posterior distributions and marginal likelihoods, and we establish their asymptotic validity. Empirically, the proposed methods are scalable and numerically stable, and for latent Gaussian models, they achieve accuracy comparable to or better than MCMC and variational competitors at substantially lower computational costs. More broadly, the framework clarifies how Laplace approximations can be justified for non-smooth generalized posteriors through local quadratic behavior of the expected loss.