On the Predictive Properties of Binary Link Functions

TL;DR

This paper rigorously compares probit and logit link functions in binary classification, establishing their predictive equivalence and exploring similarities with other link functions through theoretical and empirical analysis.

Contribution

It introduces formal definitions of structural and predictive equivalence for binary regression models and thoroughly characterizes the similarities and differences among various link functions.

Findings

Probit and logit are perfectly predictively concordant.

Cauchit and complementary log-log also show high predictive equivalence.

Empirical examples confirm theoretical equivalence results.

Abstract

This paper provides a theoretical and computational justification of the long held claim that of the similarity of the probit and logit link functions often used in binary classification. Despite this widespread recognition of the strong similarities between these two link functions, very few (if any) researchers have dedicated time to carry out a formal study aimed at establishing and characterizing firmly all the aspects of the similarities and differences. This paper proposes a definition of both structural and predictive equivalence of link functions-based binary regression models, and explores the various ways in which they are either similar or dissimilar. From a predictive analytics perspective, it turns out that not only are probit and logit perfectly predictively concordant, but the other link functions like cauchit and complementary log log enjoy very high percentage of predictive equivalence. Throughout this paper, simulated and real life examples demonstrate all the equivalence results that we prove theoretically.