Solving Sparse \& High-Dimensional-Output Regression via Compression

TL;DR

This paper introduces SHORE, a scalable sparse regression model for high-dimensional multi-output data, using compression techniques to improve interpretability and computational efficiency while maintaining accuracy.

Contribution

It proposes a novel two-stage optimization framework for sparse high-dimensional output regression that leverages compression for scalability and theoretical guarantees.

Findings

Framework is computationally scalable with provable accuracy.

Numerical results validate efficiency and accuracy.

Maintains prediction loss before and after compression.

Abstract

Multi-Output Regression (MOR) has been widely used in scientific data analysis for decision-making. Unlike traditional regression models, MOR aims to simultaneously predict multiple real-valued outputs given an input. However, the increasing dimensionality of the outputs poses significant challenges regarding interpretability and computational scalability for modern MOR applications. As a first step to address these challenges, this paper proposes a Sparse \& High-dimensional-Output REgression (SHORE) model by incorporating additional sparsity requirements to resolve the output interpretability, and then designs a computationally efficient two-stage optimization framework capable of solving SHORE with provable accuracy via compression on outputs. Theoretically, we show that the proposed framework is computationally scalable while maintaining the same order of training loss and prediction loss before-and-after compression under arbitrary or relatively weak sample set conditions. Empirically, numerical results further validate the theoretical findings, showcasing the efficiency and accuracy of the proposed framework.