# Multiple Learning for Regression in big data

## TL;DR

This paper introduces multiple learning methods using sufficient statistics arrays to efficiently compute and update multiple regression models on big data, overcoming memory limitations and reducing training time.

## Contribution

It proposes a novel memory-oblivious approach that enables simultaneous computation and updating of multiple regression models using sufficient statistics, applicable to various closed-form regressions.

## Key findings

- Achieves closed-form solutions for multiple models in big data.
- Reduces training time by half compared to traditional methods.
- Supports efficient model updates and parallel model computation.

## Abstract

Regression problems that have closed-form solutions are well understood and can be easily implemented when the dataset is small enough to be all loaded into the RAM. Challenges arise when data is too big to be stored in RAM to compute the closed form solutions. Many techniques were proposed to overcome or alleviate the memory barrier problem but the solutions are often local optimal. In addition, most approaches require accessing the raw data again when updating the models. Parallel computing clusters are also expected if multiple models need to be computed simultaneously. We propose multiple learning approaches that utilize an array of sufficient statistics (SS) to address this big data challenge. This memory oblivious approach breaks the memory barrier when computing regressions with closed-form solutions, including but not limited to linear regression, weighted linear regression, linear regression with Box-Cox transformation (Box-Cox regression) and ridge regression models. The computation and update of the SS array can be handled at per row level or per mini-batch level. And updating a model is as easy as matrix addition and subtraction. Furthermore, multiple SS arrays for different models can be easily computed simultaneously to obtain multiple models at one pass through the dataset. We implemented our approaches on Spark and evaluated over the simulated datasets. Results showed our approaches can achieve closed-form solutions of multiple models at the cost of half training time of the traditional methods for a single model.