Information Complexity Criterion for Model Selection in Robust Regression Using A New Robust Penalty Term

TL;DR

This paper introduces a new robust model selection criterion based on an information complexity approach using Huber's objective function, improving model selection accuracy in contaminated data scenarios.

Contribution

It proposes a novel information complexity criterion ($RICOMP_{C_0^{ ho_H}}$) and a new tuning parameter ($k_{C_0}$) for robust regression model selection.

Findings

$RICOMP_{C_0^{ ho_H}}$ outperforms rivals in contaminated data.

The new tuning parameter $k_{C_0}$ enhances model selection.

Simulation and real data demonstrate effectiveness.

Abstract

Model selection is basically a process of finding the best model from the subset of models in which the explanatory variables are effective on the response variable. The log likelihood function for the lack of fit term and a specified penalty term are used as two parts in a model selection criteria. In this paper, we derive a new tool for the model selection in robust regression. We introduce a new definition of relative entropy based on objective functions. Due to the analytical simplicity, we use Huber's objective function $\rho_H$ and propose our specified penalty term $C_0^{\rho_H}$ to derive new Information Complexity Criterion ($RICOMP_{C_0^{\rho_H}}$) as a robust model selection tool. Additionally, by using the properties of $C_0^{\rho_H}$, we propose a new value of tuning parameter called $k_{C_0}$ for the Huber's $\rho_H$. If a contamination to normal distribution exists, $RICOMP_{C_0^{\rho_H}}$ chooses the true model better than the rival ones. Monte Carlo Simulation studies are carried out to show the utility both of $k_{C_0}$ and $RICOMP_{C_0^{\rho_H}}$. A real data example is also given.