Convolutional Networks with Adaptive Inference Graphs

TL;DR

ConvNet-AIG introduces adaptive inference graphs that dynamically select layers based on input, improving efficiency, accuracy, and robustness over traditional fixed-structure convolutional networks.

Contribution

This work presents a novel convolutional network architecture with input-dependent topology, enabling dynamic layer selection to enhance performance and robustness.

Findings

Outperforms ResNet with fewer computations on ImageNet.

Learns distinct inference graphs for different categories.

Shows increased robustness to adversarial examples.

Abstract

Do convolutional networks really need a fixed feed-forward structure? What if, after identifying the high-level concept of an image, a network could move directly to a layer that can distinguish fine-grained differences? Currently, a network would first need to execute sometimes hundreds of intermediate layers that specialize in unrelated aspects. Ideally, the more a network already knows about an image, the better it should be at deciding which layer to compute next. In this work, we propose convolutional networks with adaptive inference graphs (ConvNet-AIG) that adaptively define their network topology conditioned on the input image. Following a high-level structure similar to residual networks (ResNets), ConvNet-AIG decides for each input image on the fly which layers are needed. In experiments on ImageNet we show that ConvNet-AIG learns distinct inference graphs for different categories. Both ConvNet-AIG with 50 and 101 layers outperform their ResNet counterpart, while using 20% and 38% less computations respectively. By grouping parameters into layers for related classes and only executing relevant layers, ConvNet-AIG improves both efficiency and overall classification quality. Lastly, we also study the effect of adaptive inference graphs on the susceptibility towards adversarial examples. We observe that ConvNet-AIG shows a higher robustness than ResNets, complementing other known defense mechanisms.