Gaussian Regularized Sliced Inverse Regression

TL;DR

This paper introduces a Gaussian prior-based regularization framework for Sliced Inverse Regression, addressing issues of collinearity and small sample sizes, and proposes new regularization methods with comparative simulation results.

Contribution

It provides a unified framework for existing SIR regularizations and introduces three novel priors for improved dimension reduction in high-dimensional data.

Findings

Existing regularizations fit into the proposed framework

Three new regularization methods are proposed

Simulation results demonstrate effectiveness of new methods

Abstract

Sliced Inverse Regression (SIR) is an effective method for dimension reduction in high-dimensional regression problems. The original method, however, requires the inversion of the predictors covariance matrix. In case of collinearity between these predictors or small sample sizes compared to the dimension, the inversion is not possible and a regularization technique has to be used. Our approach is based on a Fisher Lecture given by R.D. Cook where it is shown that SIR axes can be interpreted as solutions of an inverse regression problem. In this paper, a Gaussian prior distribution is introduced on the unknown parameters of the inverse regression problem in order to regularize their estimation. We show that some existing SIR regularizations can enter our framework, which permits a global understanding of these methods. Three new priors are proposed leading to new regularizations of the SIR method. A comparison on simulated data is provided.