Distributed Bayesian Piecewise Sparse Linear Models

TL;DR

This paper introduces a distributed Bayesian inference method for piecewise sparse linear models, enhancing interpretability and scalability for large datasets while maintaining high accuracy.

Contribution

It presents a novel distributed FAB inference approach that enables efficient model selection and scalability without extensive data communication.

Findings

Achieves high prediction accuracy on synthetic and benchmark datasets.

Demonstrates linear scale-out with increasing CPU cores.

Maintains interpretability of piecewise linear models.

Abstract

The importance of interpretability of machine learning models has been increasing due to emerging enterprise predictive analytics, threat of data privacy, accountability of artificial intelligence in society, and so on. Piecewise linear models have been actively studied to achieve both accuracy and interpretability. They often produce competitive accuracy against state-of-the-art non-linear methods. In addition, their representations (i.e., rule-based segmentation plus sparse linear formula) are often preferred by domain experts. A disadvantage of such models, however, is high computational cost for simultaneous determinations of the number of "pieces" and cardinality of each linear predictor, which has restricted their applicability to middle-scale data sets. This paper proposes a distributed factorized asymptotic Bayesian (FAB) inference of learning piece-wise sparse linear models on distributed memory architectures. The distributed FAB inference solves the simultaneous model selection issue without communicating $O(N)$ data where N is the number of training samples and achieves linear scale-out against the number of CPU cores. Experimental results demonstrate that the distributed FAB inference achieves high prediction accuracy and performance scalability with both synthetic and benchmark data.