Dynamic Routing Networks

TL;DR

Dynamic Routing Networks (DRNets) enable instance-aware inference by dynamically routing inputs through selected branches, significantly reducing computational costs while maintaining high prediction accuracy.

Contribution

The paper introduces DRNets, a novel architecture that dynamically routes inputs through selected branches for efficient inference, addressing the inefficiency of static models.

Findings

DRNets reduce parameters and FLOPs during inference.

DRNets achieve comparable accuracy to state-of-the-art models.

Dynamic routing improves inference efficiency significantly.

Abstract

The deployment of deep neural networks in real-world applications is mostly restricted by their high inference costs. Extensive efforts have been made to improve the accuracy with expert-designed or algorithm-searched architectures. However, the incremental improvement is typically achieved with increasingly more expensive models that only a small portion of input instances really need. Inference with a static architecture that processes all input instances via the same transformation would thus incur unnecessary computational costs. Therefore, customizing the model capacity in an instance-aware manner is much needed for higher inference efficiency. In this paper, we propose Dynamic Routing Networks (DRNets), which support efficient instance-aware inference by routing the input instance to only necessary transformation branches selected from a candidate set of branches for each connection between transformation nodes. The branch selection is dynamically determined via the corresponding branch importance weights, which are first generated from lightweight hypernetworks (RouterNets) and then recalibrated with Gumbel-Softmax before the selection. Extensive experiments show that DRNets can reduce a substantial amount of parameter size and FLOPs during inference with prediction performance comparable to state-of-the-art architectures.