Smoothing spline ANOVA for super-large samples: Scalable computation via rounding parameters

TL;DR

This paper introduces a scalable algorithm for smoothing spline ANOVA models that uses rounding parameters, enabling efficient analysis of super-large datasets within seconds on standard computers.

Contribution

The paper proposes a novel algorithm incorporating rounding parameters to significantly reduce computational costs of SSANOVA for large datasets.

Findings

Fitting SSANOVA models to large data is feasible within seconds.

Rounding parameters maintain accuracy while improving scalability.

Method demonstrated on EEG data with successful results.

Abstract

In the current era of big data, researchers routinely collect and analyze data of super-large sample sizes. Data-oriented statistical methods have been developed to extract information from super-large data. Smoothing spline ANOVA (SSANOVA) is a promising approach for extracting information from noisy data; however, the heavy computational cost of SSANOVA hinders its wide application. In this paper, we propose a new algorithm for fitting SSANOVA models to super-large sample data. In this algorithm, we introduce rounding parameters to make the computation scalable. To demonstrate the benefits of the rounding parameters, we present a simulation study and a real data example using electroencephalography data. Our results reveal that (using the rounding parameters) a researcher can fit nonparametric regression models to very large samples within a few seconds using a standard laptop or tablet computer.