Multinomial Logistic Regression: Asymptotic Normality on Null Covariates in High-Dimensions

TL;DR

This paper studies the asymptotic behavior of the multinomial logistic regression MLE in high-dimensional settings, especially for null covariates, and proposes a new significance testing methodology validated through simulations.

Contribution

It extends asymptotic normality results to multinomial logistic models in high dimensions and introduces a novel feature significance test.

Findings

Asymptotic normality holds for the multinomial logistic MLE on null covariates in high dimensions.

The proposed p-value methodology accurately tests feature significance, validated by simulations.

The theory generalizes classical results to multi-class high-dimensional settings.

Abstract

This paper investigates the asymptotic distribution of the maximum-likelihood estimate (MLE) in multinomial logistic models in the high-dimensional regime where dimension and sample size are of the same order. While classical large-sample theory provides asymptotic normality of the MLE under certain conditions, such classical results are expected to fail in high-dimensions as documented for the binary logistic case in the seminal work of Sur and Cand\`es [2019]. We address this issue in classification problems with 3 or more classes, by developing asymptotic normality and asymptotic chi-square results for the multinomial logistic MLE (also known as cross-entropy minimizer) on null covariates. Our theory leads to a new methodology to test the significance of a given feature. Extensive simulation studies on synthetic data corroborate these asymptotic results and confirm the validity of proposed p-values for testing the significance of a given feature.