Building a GPU-Accelerated Multivariate Statistics Platform

TL;DR

This paper demonstrates how to efficiently scale classical multivariate statistical methods like covariance estimation and PCA on billion-row datasets using a GPU-accelerated workflow on a single multi-GPU node.

Contribution

It introduces a GPU-based approach for computing sufficient statistics in a single pass over massive data, enabling scalable multivariate analysis.

Findings

Achieved single-pass computation of statistics on 10-billion-row dataset

Highlighted importance of data representation and numerical stability at large scale

Validated methods using known invariants for correctness

Abstract

Classical multivariate statistical methods such as covariance estimation and principal component analysis are well understood mathematically, yet their application at extreme data scales remains challenging. When the number of observations reaches billions, performance is limited by data movement, input-output bottlenecks, and numerical stability rather than arithmetic complexity. This work presents a case study of scaling classical multivariate statistics on a single multi-GPU node. Using C++ and CUDA, a GPU-accelerated workflow was developed to compute sufficient statistics in a single pass over a 10-billion-row dataset. Column sums and cross-product matrices are used to enable downstream computation of means, covariance, correlation, and principal component analysis without revisiting the raw data. The results highlight the importance of data representation, validation using known invariants, and careful numerical treatment when applying established statistical methods at large scale.