DJEnsemble: On the Selection of a Disjoint Ensemble of Deep Learning Black-Box Spatio-Temporal Models

TL;DR

This paper introduces DJEnsemble, a cost-based method for automatically selecting and allocating disjoint ensembles of black-box deep learning models for spatio-temporal prediction, significantly improving efficiency and accuracy.

Contribution

It presents the first optimization approach for allocating black-box models in spatio-temporal ensembles, combining offline preprocessing with online cost minimization.

Findings

Achieves up to 4X faster execution time compared to traditional methods.

Improves prediction accuracy by nearly 9X over baseline ensemble approaches.

Cost model closely approximates the optimal plan in experiments.

Abstract

In this paper, we present a cost-based approach for the automatic selection and allocation of a disjoint ensemble of black-box predictors to answer predictive spatio-temporal queries. Our approach is divided into two parts -- offline and online. During the offline part, we preprocess the predictive domain data -- transforming it into a regular grid -- and the black-box models -- computing their spatio-temporal learning function. In the online part, we compute a DJEnsemble plan which minimizes a multivariate cost function based on estimates for the prediction error and the execution cost -- producing a model spatial allocation matrix -- and run the optimal ensemble plan. We conduct a set of extensive experiments that evaluate the DJEnsemble approach and highlight its efficiency. We show that our cost model produces plans with performance close to the actual best plan. When compared against the traditional ensemble approach, DJEnsemble achieves up to $4X$ improvement in execution time and almost $9X$ improvement in prediction accuracy. To the best of our knowledge, this is the first work to solve the problem of optimizing the allocation of black-box models to answer predictive spatio-temporal queries.