Feature Embedding by Template Matching as a ResNet Block

TL;DR

This paper reinterprets ResNet blocks as template matching for local feature embedding and introduces a residual block that explicitly incorporates label information to enhance semantic feature extraction, improving classification performance.

Contribution

It presents a novel perspective of ResNet blocks as template matching and designs a new residual block that explicitly embeds semantic features using label information.

Findings

Consistent performance improvements on benchmark datasets

Enhanced local feature embedding through label-guided residual blocks

Reinterpretation of convolution as template matching

Abstract

Convolution blocks serve as local feature extractors and are the key to success of the neural networks. To make local semantic feature embedding rather explicit, we reformulate convolution blocks as feature selection according to the best matching kernel. In this manner, we show that typical ResNet blocks indeed perform local feature embedding via template matching once batch normalization (BN) followed by a rectified linear unit (ReLU) is interpreted as arg-max optimizer. Following this perspective, we tailor a residual block that explicitly forces semantically meaningful local feature embedding through using label information. Specifically, we assign a feature vector to each local region according to the classes that the corresponding region matches. We evaluate our method on three popular benchmark datasets with several architectures for image classification and consistently show that our approach substantially improves the performance of the baseline architectures.