An automatic robust Bayesian approach to principal component regression

TL;DR

This paper introduces a robust Bayesian principal component regression method that automatically downweights outliers in high-dimensional data, improving prediction accuracy and model reliability.

Contribution

The paper presents a novel Bayesian approach that achieves outlier robustness in principal component regression, with automatic penalization and model-averaging for uncertainty.

Findings

Outperforms nonrobust Bayesian and frequentist methods on real data.

Automatically penalizes outliers, ensuring predictions align with the main data trend.

Provides automated procedures and code for practical implementation.

Abstract

Principal component regression uses principal components as regressors. It is particularly useful in prediction settings with high-dimensional covariates. The existing literature treating of Bayesian approaches is relatively sparse. We introduce a Bayesian approach that is robust to outliers in both the dependent variable and the covariates. Outliers can be thought of as observations that are not in line with the general trend. The proposed approach automatically penalises these observations so that their impact on the posterior gradually vanishes as they move further and further away from the general trend, corresponding to a concept in Bayesian statistics called whole robustness. The predictions produced are thus consistent with the bulk of the data. The approach also exploits the geometry of principal components to efficiently identify those that are significant. Individual predictions obtained from the resulting models are consolidated according to model-averaging mechanisms to account for model uncertainty. The approach is evaluated on real data and compared to its nonrobust Bayesian counterpart, the traditional frequentist approach, and a commonly employed robust frequentist method. Detailed guidelines to automate the entire statistical procedure are provided. All required code is made available, see ArXiv:1711.06341.