Efficient Computation for Centered Linear Regression with Sparse Inputs

TL;DR

This paper introduces an efficient method for centered linear regression with sparse inputs that enhances computational speed and reduces memory usage by exploiting data sparsity despite the challenges posed by centering.

Contribution

The paper presents a novel approach to perform centered linear regression efficiently on sparse data without densifying the input, improving speed and memory efficiency.

Findings

Significant reduction in computation time compared to traditional methods.

Lower memory footprint for large-scale sparse data.

Maintains accuracy of regression estimates with the proposed method.

Abstract

Regression with sparse inputs is a common theme for large scale models. Optimizing the underlying linear algebra for sparse inputs allows such models to be estimated faster. At the same time, centering the inputs has benefits in improving the interpretation and convergence of the model. However, centering the data naturally makes sparse data become dense, limiting opportunities for optimization. We propose an efficient strategy that estimates centered regression while taking advantage of sparse structure in data, improving computational performance and decreasing the memory footprint of the estimator.