# Compressed and Penalized Linear Regression

## TL;DR

This paper introduces new methods for linear regression that balance computational efficiency and statistical accuracy, leveraging approximation and regularization techniques to improve estimation and prediction in large datasets.

## Contribution

It proposes novel regularized linear regression procedures inspired by approximation theory, demonstrating their benefits for both computation and statistical performance.

## Key findings

- Methods improve computational speed on large datasets.
- Regularization enhances statistical accuracy and interpretability.
- Approximations can outperform exact solutions in statistical tasks.

## Abstract

Modern applications require methods that are computationally feasible on large datasets but also preserve statistical efficiency. Frequently, these two concerns are seen as contradictory: approximation methods that enable computation are assumed to degrade statistical performance relative to exact methods. In applied mathematics, where much of the current theoretical work on approximation resides, the inputs are considered to be observed exactly. The prevailing philosophy is that while the exact problem is, regrettably, unsolvable, any approximation should be as small as possible. However, from a statistical perspective, an approximate or regularized solution may be preferable to the exact one. Regularization formalizes a trade-off between fidelity to the data and adherence to prior knowledge about the data-generating process such as smoothness or sparsity. The resulting estimator tends to be more useful, interpretable, and suitable as an input to other methods. In this paper, we propose new methodology for estimation and prediction under a linear model borrowing insights from the approximation literature. We explore these procedures from a statistical perspective and find that in many cases they improve both computational and statistical performance.