Computationally Efficient Estimation of Large Probit Models

TL;DR

This paper introduces a fast, deterministic EM algorithm using expectation propagation for estimating large probit models, enabling analysis of high-dimensional correlated discrete data efficiently.

Contribution

It develops a novel EM algorithm that leverages expectation propagation and a symmetry-based transformation to efficiently estimate large probit models with many alternatives.

Findings

Enables analysis of over 100 alternatives in probit models.

Provides a deterministic alternative to sampling-based methods.

Applied to hotel search data to reveal ordering effects.

Abstract

Probit models are useful for modeling correlated discrete responses in many disciplines, including consumer choice data in economics and marketing. However, the Gaussian latent variable feature of probit models coupled with identification constraints pose significant computational challenges for its estimation and inference, especially when the dimension of the discrete response variable is large. In this paper, we propose a computationally efficient Expectation-Maximization (EM) algorithm for estimating large probit models. Our work is distinct from existing methods in two important aspects. First, instead of simulation or sampling methods, we apply and customize expectation propagation (EP), a deterministic method originally proposed for approximate Bayesian inference, to estimate moments of the truncated multivariate normal (TMVN) in the E (expectation) step. Second, we take advantage of a symmetric identification condition to transform the constrained optimization problem in the M (maximization) step into a one-dimensional problem, which is solved efficiently using Newton's method instead of off-the-shelf solvers. Our method enables the analysis of correlated choice data in the presence of more than 100 alternatives, which is a reasonable size in modern applications, such as online shopping and booking platforms, but has been difficult in practice with probit models. We apply our probit estimation method to study ordering effects in hotel search results on Expedia's online booking platform.