A Minimum Message Length Criterion for Robust Linear Regression

TL;DR

This paper introduces a novel minimum message length-based criterion for robust linear regression with Student-t errors, offering a hyperparameter-free, invariant, and effective method for variable selection and parameter estimation.

Contribution

It develops a new MML-based criterion for Student-t regression that is simple, hyperparameter-free, and invariant, improving model selection robustness.

Findings

Performs well in simulations and real data comparisons.

Outperforms AIC and BIC in robustness and accuracy.

Provides a practical, easy-to-apply variable selection method.

Abstract

This paper applies the minimum message length principle to inference of linear regression models with Student-t errors. A new criterion for variable selection and parameter estimation in Student-t regression is proposed. By exploiting properties of the regression model, we derive a suitable non-informative proper uniform prior distribution for the regression coefficients that leads to a simple and easy-to-apply criterion. Our proposed criterion does not require specification of hyperparameters and is invariant under both full rank transformations of the design matrix and linear transformations of the outcomes. We compare the proposed criterion with several standard model selection criteria, such as the Akaike information criterion and the Bayesian information criterion, on simulations and real data with promising results.