Approximation Algorithms for Cascading Prediction Models

TL;DR

This paper introduces an approximation algorithm that creates cascaded prediction models from pre-trained models, significantly reducing computational cost and memory I/O while maintaining accuracy, demonstrated on ImageNet classification.

Contribution

It presents a novel approximation algorithm for constructing cost-efficient cascaded models from existing pre-trained models, optimizing for accuracy and computational efficiency.

Findings

Up to 2x reduction in floating point multiplications.

Up to 6x reduction in average-case memory I/O.

Cascades adapt input resolution and confidence thresholds based on image difficulty.

Abstract

We present an approximation algorithm that takes a pool of pre-trained models as input and produces from it a cascaded model with similar accuracy but lower average-case cost. Applied to state-of-the-art ImageNet classification models, this yields up to a 2x reduction in floating point multiplications, and up to a 6x reduction in average-case memory I/O. The auto-generated cascades exhibit intuitive properties, such as using lower-resolution input for easier images and requiring higher prediction confidence when using a computationally cheaper model.