COMET: A Recipe for Learning and Using Large Ensembles on Massive Data

TL;DR

COMET introduces a scalable MapReduce algorithm for training large random forest ensembles on massive datasets, improving accuracy and efficiency through IVoting and a novel lazy evaluation method.

Contribution

It presents a single-pass distributed method for large-scale ensemble learning and a new Gaussian-based lazy evaluation technique to reduce inference costs.

Findings

COMET achieves better accuracy and training time compared to serial subsampling methods.

IVoting enhances ensemble accuracy over traditional bagging.

Lazy ensemble evaluation reduces inference cost by over 100 times.

Abstract

COMET is a single-pass MapReduce algorithm for learning on large-scale data. It builds multiple random forest ensembles on distributed blocks of data and merges them into a mega-ensemble. This approach is appropriate when learning from massive-scale data that is too large to fit on a single machine. To get the best accuracy, IVoting should be used instead of bagging to generate the training subset for each decision tree in the random forest. Experiments with two large datasets (5GB and 50GB compressed) show that COMET compares favorably (in both accuracy and training time) to learning on a subsample of data using a serial algorithm. Finally, we propose a new Gaussian approach for lazy ensemble evaluation which dynamically decides how many ensemble members to evaluate per data point; this can reduce evaluation cost by 100X or more.