Large multi-response linear regression estimation based on low-rank pre-smoothing

TL;DR

This paper introduces a low-rank pre-smoothing method for large multi-response linear regression, improving estimation accuracy and computational efficiency over traditional methods.

Contribution

It extends pre-smoothing techniques to multi-response settings using low-rank approximation, with theoretical guarantees and empirical validation.

Findings

Outperforms ordinary least squares in mean squared error in large-response scenarios.

More computationally efficient than reduced rank regression.

Demonstrated effectiveness on environmental and biological data.

Abstract

Pre-smoothing is a technique aimed at increasing the signal-to-noise ratio in data to improve subsequent estimation and model selection in regression problems. However, pre-smoothing has thus far been limited to the univariate response regression setting. However, there are many scientific applications in which interest lies in multi-response regression problems, particularly when the number of responses is large. Motivated by this setting, this article proposes a technique for data pre-smoothing based on low-rank approximation. We establish theoretical results on the performance of the proposed methodology, which show that in this large-response setting, the proposed technique outperforms ordinary least squares estimation with the mean squared error criterion, whilst being computationally more efficient than alternative approaches such as reduced rank regression. We quantify our estimator's benefit empirically in a number of simulated experiments. We also demonstrate our proposed low-rank pre-smoothing technique on real data arising from the environmental and biological sciences.