Posterior Convergence of Nonparametric Binary and Poisson Regression Under Possible Misspecifications

TL;DR

This paper studies the convergence behavior of nonparametric binary and Poisson regression models' posteriors under potential misspecification, establishing consistency, convergence rates, and predictive accuracy using stochastic process priors.

Contribution

It introduces a novel approach combining asymptotic equipartition property and sieve methods for posterior convergence analysis under model misspecification.

Findings

Posterior consistency is established for both binary and Poisson regression.

Convergence rates are characterized by the Kullback-Leibler divergence rate.

Posterior predictive distribution can approximate the true distribution despite misspecification.

Abstract

In this article, we investigate posterior convergence of nonparametric binary and Poisson regression under possible model misspecification, assuming general stochastic process prior with appropriate properties. Our model setup and objective for binary regression is similar to that of Ghosal and Roy (2006) where the authors have used the approach of entropy bound and exponentially consistent tests with the sieve method to achieve consistency with respect to their Gaussian process prior. In contrast, for both binary and Poisson regression, using general stochastic process prior, our approach involves verification of asymptotic equipartition property along with the method of sieve, which is a manoeuvre of the general results of Shalizi (2009), useful even for misspecified models. Moreover, we will establish not only posterior consistency but also the rates at which the posterior probabilities converge, which turns out to be the Kullback-Leibler divergence rate. We also investgate the traditional posterior convergence rates. Interestingly, from subjective Bayesian viewpoint we will show that the posterior predictive distribution can accurately approximate the best possible predictive distribution in the sense that the Hellinger distance, as well as the total variation distance between the two distributions can tend to zero, in spite of misspecifications.