Grouped fixed effects regularization for binary choice models

TL;DR

This paper introduces a grouped fixed effects regularization method for binary choice panel data models, addressing issues of complete separation and bias in estimates by clustering unobserved heterogeneity.

Contribution

It proposes a novel clustering-based regularization approach that improves estimation accuracy and inference in binary choice models with panel data and severe separation.

Findings

Unbiased estimates and reliable inference for Average Partial Effects achieved.

The method reduces bias caused by complete separation in binary panel data.

Empirical applications demonstrate improved forecasting and sensitivity analysis.

Abstract

We study the application of the grouped fixed effects approach to binary choice models for panel data in presence of severe complete separation. Through data loss, complete separation may lead to biased estimates of Average Partial Effects and imprecise inference. Moreover, forecasts are not available for units without variability in the response configuration. The grouped fixed effects approach discretizes unobserved heterogeneity via k-means clustering, thus reducing the number of fixed effects to estimate. This regularization reduces complete separation, since it relies on within-cluster rather than within-subject response transitions. Drawing from asymptotic theory for the APEs, we propose choosing a number of groups such that clustering delivers a good approximation of the latent trait while keeping the incidental parameters problem under control. The simulation results show that the proposed approach delivers unbiased estimates and reliable inference for the APEs. Two empirical applications illustrate the sensitivity of the results to the choice of the number of groups and how nontrivial forecasts for a much larger number of units can be obtained.