Bayesian Inference for Generalized Linear Model with Linear Inequality Constraints

TL;DR

This paper introduces a Bayesian inference method for Generalized Linear Models with linear inequality constraints, utilizing a generalized truncated normal prior and a product slice sampler for efficient, constrained parameter estimation.

Contribution

It develops a novel Bayesian approach with a specialized prior and a uniformly ergodic slice sampling algorithm for constrained GLMs, applicable to various models like logistic and Poisson regressions.

Findings

The proposed method effectively quantifies uncertainty in constrained parameters.

Numerical simulations show reduced bias and variance compared to existing methods.

Real case studies demonstrate practical applicability and improved inference accuracy.

Abstract

Bayesian statistical inference for Generalized Linear Models (GLMs) with parameters lying on a constrained space is of general interest (e.g., in monotonic or convex regression), but often constructing valid prior distributions supported on a subspace spanned by a set of linear inequality constraints can be challenging, especially when some of the constraints might be binding leading to a lower dimensional subspace. For the general case with canonical link, it is shown that a generalized truncated multivariate normal supported on a desired subspace can be used. Moreover, it is shown that such prior distribution facilitates the construction of a general purpose product slice sampling method to obtain (approximate) samples from corresponding posterior distribution, making the inferential method computationally efficient for a wide class of GLMs with an arbitrary set of linear inequality constraints. The proposed product slice sampler is shown to be uniformly ergodic, having a geometric convergence rate under a set of mild regularity conditions satisfied by many popular GLMs (e.g., logistic and Poisson regressions with constrained coefficients). One of the primary advantages of the proposed Bayesian estimation method over classical methods is that uncertainty of parameter estimates is easily quantified by using the samples simulated from the path of the Markov Chain of the slice sampler. Numerical illustrations using simulated data sets are presented to illustrate the superiority of the proposed methods compared to some existing methods in terms of sampling bias and variances. In addition, real case studies are presented using data sets for fertilizer-crop production and estimating the SCRAM rate in nuclear power plants.