Divide and Conquer Local Average Regression

TL;DR

This paper combines divide and conquer strategies with local average regression to efficiently analyze massive datasets, proposing variants that achieve optimal learning rates with fewer restrictions.

Contribution

It introduces two variants of divide and conquer local average regression that relax data block restrictions while maintaining optimal learning rates.

Findings

The original method reaches the optimal learning rate but with strong restrictions.

The proposed variants reduce or eliminate restrictions, still achieving optimal learning rates.

Experimental results verify the theoretical advantages of the variants.

Abstract

The divide and conquer strategy, which breaks a massive data set into a se- ries of manageable data blocks, and then combines the independent results of data blocks to obtain a final decision, has been recognized as a state-of-the-art method to overcome challenges of massive data analysis. In this paper, we merge the divide and conquer strategy with local average regression methods to infer the regressive relationship of input-output pairs from a massive data set. After theoretically analyzing the pros and cons, we find that although the divide and conquer local average regression can reach the optimal learning rate, the restric- tion to the number of data blocks is a bit strong, which makes it only feasible for small number of data blocks. We then propose two variants to lessen (or remove) this restriction. Our results show that these variants can achieve the optimal learning rate with much milder restriction (or without such restriction). Extensive experimental studies are carried out to verify our theoretical assertions.