Stochastic Newton Sampler: R Package sns

TL;DR

The sns R package implements a Stochastic Newton Sampler algorithm that uses local second-order geometry for efficient MCMC sampling, especially in high-dimensional Gaussian-like models, with convergence and diagnostic tools.

Contribution

Introduction of the sns R package implementing a Stochastic Newton Sampler with strategies for improved convergence and mixing in high-dimensional Bayesian models.

Findings

SNS approaches independent sampling in Gaussian-like densities.

Partitioning enhances mixing in high-dimensional spaces.

Diagnostics and visualization tools support Bayesian analysis.

Abstract

The R package sns implements Stochastic Newton Sampler (SNS), a Metropolis-Hastings Monte Carlo Markov Chain algorithm where the proposal density function is a multivariate Gaussian based on a local, second-order Taylor series expansion of log-density. The mean of the proposal function is the full Newton step in Newton-Raphson optimization algorithm. Taking advantage of the local, multivariate geometry captured in log-density Hessian allows SNS to be more efficient than univariate samplers, approaching independent sampling as the density function increasingly resembles a multivariate Gaussian. SNS requires the log-density Hessian to be negative-definite everywhere in order to construct a valid proposal function. This property holds, or can be easily checked, for many GLM-like models. When initial point is far from density peak, running SNS in non-stochastic mode by taking the Newton step, augmented with with line search, allows the MCMC chain to converge to high-density areas faster. For high-dimensional problems, partitioning of state space into lower-dimensional subsets, and applying SNS to the subsets within a Gibbs sampling framework can significantly improve the mixing of SNS chains. In addition to the above strategies for improving convergence and mixing, sns offers diagnostics and visualization capabilities, as well as a function for sample-based calculation of Bayesian predictive posterior distributions.