On the contraction properties of some high-dimensional quasi-posterior distributions

TL;DR

This paper investigates the contraction properties of high-dimensional quasi-posterior distributions, establishing conditions for sparsity and applying results to logistic regression and binary graphical models, with specific contraction rate bounds.

Contribution

It provides general theoretical results on the contraction of quasi-posteriors in high dimensions and applies these to logistic and graphical models, deriving explicit convergence rates.

Findings

Quasi-posterior concentrates on sparse parameter subsets.

Contraction rate for logistic regression: O(√(s*log(d)/n)).

Contraction rate for graphical models: O(√((p+S)log(p)/n)).

Abstract

We study the contraction properties of a quasi-posterior distribution $\check\Pi_{n,d}$ obtained by combining a quasi-likelihood function and a sparsity inducing prior distribution on $\rset^d$, as both $n$ (the sample size), and $d$ (the dimension of the parameter) increase. We derive some general results that highlight a set of sufficient conditions under which $\check\Pi_{n,d}$ puts increasingly high probability on sparse subsets of $\rset^d$, and contracts towards the true value of the parameter. We apply these results to the analysis of logistic regression models, and binary graphical models, in high-dimensional settings. For the logistic regression model, we shows that for well-behaved design matrices, the posterior distribution contracts at the rate $O(\sqrt{s_\star\log(d)/n})$, where $s_\star$ is the number of non-zero components of the parameter. For the binary graphical model, under some regularity conditions, we show that a quasi-posterior analog of the neighborhood selection of \cite{meinshausen06} contracts in the Frobenius norm at the rate $O(\sqrt{(p+S)\log(p)/n})$, where $p$ is the number of nodes, and $S$ the number of edges of the true graph.