Linearized Binary Regression

TL;DR

This paper introduces a linearized approach to binary regression that matches the performance of nonlinear methods, providing exact error analysis and broad applicability in various fields dealing with binary data.

Contribution

It demonstrates that linearizing the probit model with linear estimators achieves competitive results and offers exact, nonasymptotic error expressions, simplifying analysis compared to nonlinear methods.

Findings

Linearized binary regression performs on par with nonlinear methods.

Exact formulas for mean-squared error are derived.

Effective on synthetic and real-world datasets.

Abstract

Probit regression was first proposed by Bliss in 1934 to study mortality rates of insects. Since then, an extensive body of work has analyzed and used probit or related binary regression methods (such as logistic regression) in numerous applications and fields. This paper provides a fresh angle to such well-established binary regression methods. Concretely, we demonstrate that linearizing the probit model in combination with linear estimators performs on par with state-of-the-art nonlinear regression methods, such as posterior mean or maximum aposteriori estimation, for a broad range of real-world regression problems. We derive exact, closed-form, and nonasymptotic expressions for the mean-squared error of our linearized estimators, which clearly separates them from nonlinear regression methods that are typically difficult to analyze. We showcase the efficacy of our methods and results for a number of synthetic and real-world datasets, which demonstrates that linearized binary regression finds potential use in a variety of inference, estimation, signal processing, and machine learning applications that deal with binary-valued observations or measurements.