Window-Based Early-Exit Cascades for Uncertainty Estimation: When Deep Ensembles are More Efficient than Single Models

TL;DR

This paper introduces a window-based early-exit cascade method for deep ensembles that improves uncertainty estimation efficiency, outperforming single models in computational cost while maintaining accuracy, especially on out-of-distribution data.

Contribution

The work extends early-exit cascade techniques to uncertainty tasks by selectively passing samples near decision boundaries, demonstrating improved efficiency and reliability over scaled single models.

Findings

Cascaded ensembles achieve similar uncertainty coverage with fewer MACs.

The approach outperforms scaled single models in OOD uncertainty estimation.

EfficientNet-B2 ensemble matches EfficientNet-B4 performance at 30% of MACs.

Abstract

Deep Ensembles are a simple, reliable, and effective method of improving both the predictive performance and uncertainty estimates of deep learning approaches. However, they are widely criticised as being computationally expensive, due to the need to deploy multiple independent models. Recent work has challenged this view, showing that for predictive accuracy, ensembles can be more computationally efficient (at inference) than scaling single models within an architecture family. This is achieved by cascading ensemble members via an early-exit approach. In this work, we investigate extending these efficiency gains to tasks related to uncertainty estimation. As many such tasks, e.g. selective classification, are binary classification, our key novel insight is to only pass samples within a window close to the binary decision boundary to later cascade stages. Experiments on ImageNet-scale data across a number of network architectures and uncertainty tasks show that the proposed window-based early-exit approach is able to achieve a superior uncertainty-computation trade-off compared to scaling single models. For example, a cascaded EfficientNet-B2 ensemble is able to achieve similar coverage at 5% risk as a single EfficientNet-B4 with <30% the number of MACs. We also find that cascades/ensembles give more reliable improvements on OOD data vs scaling models up. Code for this work is available at: https://github.com/Guoxoug/window-early-exit.