Interpreting the Residual Stream of ResNet18

TL;DR

This paper explores the residual stream in ResNet18, revealing how it manages features and implements scale invariance, advancing understanding of deep neural network computations in visual recognition.

Contribution

It provides the first detailed mechanistic analysis of the residual stream in ResNet18, highlighting its role in feature management and scale invariance.

Findings

Residual stream channels perform various updates from skipping to overwriting features.

Many channels compute scale-invariant representations by mixing features.

The residual stream acts as a flexible feature manager and supports scale invariance.

Abstract

A mechanistic understanding of the computations learned by deep neural networks (DNNs) is far from complete. In the domain of visual object recognition, prior research has illuminated inner workings of InceptionV1, but DNNs with different architectures have remained largely unexplored. This work investigates ResNet18 with a particular focus on its residual stream, an architectural mechanism which InceptionV1 lacks. We observe that for a given block, channel features of the stream are updated along a spectrum: either the input feature skips to the output, the block feature overwrites the output, or the output is some mixture between the input and block features. Furthermore, we show that many residual stream channels compute scale invariant representations through a mixture of the input's smaller-scale feature with the block's larger-scale feature. This not only mounts evidence for the universality of scale equivariance, but also presents how the residual stream further implements scale invariance. Collectively, our results begin an interpretation of the residual stream in visual object recognition, finding it to be a flexible feature manager and a medium to build scale invariant representations.