Regression model selection via log-likelihood ratio and constrained minimum criterion

TL;DR

This paper introduces a novel regression model selection method based on the log-likelihood ratio, which effectively identifies the true model with high probability, outperforming traditional criteria like AIC and BIC.

Contribution

The paper develops a new model selection approach using the likelihood ratio test set, providing theoretical guarantees and demonstrating superior accuracy over existing criteria.

Findings

High probability of selecting the true model with large samples and small significance levels.

Method outperforms AIC and BIC in simulation studies.

Successfully applied to logistic regression for heart disease data.

Abstract

Although the log-likelihood is widely used in model selection, the log-likelihood ratio has had few applications in this area. We develop a log-likelihood ratio based method for selecting regression models by focusing on the set of models deemed plausible by the likelihood ratio test. We show that when the sample size is large and the significance level of the test is small, there is a high probability that the smallest model in the set is the true model; thus, we select this smallest model. The significance level of the test serves as a parameter for this method. We consider three levels of this parameter in a simulation study and compare this method with the Akaike Information Criterion and Bayesian Information Criterion to demonstrate its excellent accuracy and adaptability to different sample sizes. We also apply this method to select a logistic regression model for a South African heart disease dataset.