Impatient DNNs - Deep Neural Networks with Dynamic Time Budgets

TL;DR

This paper introduces Impatient DNNs, a framework enabling deep neural networks to adaptively provide predictions at multiple stages based on individual time budgets, improving efficiency for real-time applications.

Contribution

It presents a novel framework for training DNNs with dynamic, learnable early predictors that jointly optimize for accuracy under varying time constraints.

Findings

Significant accuracy gains over baseline methods.

Effective handling of diverse budget distributions.

Applicable across different architectures and datasets.

Abstract

We propose Impatient Deep Neural Networks (DNNs) which deal with dynamic time budgets during application. They allow for individual budgets given a priori for each test example and for anytime prediction, i.e., a possible interruption at multiple stages during inference while still providing output estimates. Our approach can therefore tackle the computational costs and energy demands of DNNs in an adaptive manner, a property essential for real-time applications. Our Impatient DNNs are based on a new general framework of learning dynamic budget predictors using risk minimization, which can be applied to current DNN architectures by adding early prediction and additional loss layers. A key aspect of our method is that all of the intermediate predictors are learned jointly. In experiments, we evaluate our approach for different budget distributions, architectures, and datasets. Our results show a significant gain in expected accuracy compared to common baselines.