Targeted Kernel Networks: Faster Convolutions with Attentive Regularization

TL;DR

This paper introduces Targeted Kernel Networks (TKNs), which use Attentive Regularization to focus kernels on specific regions, reducing computation and improving performance in CNNs.

Contribution

The paper presents a novel Attentive Regularization method that constrains kernel activation regions, enabling faster CNNs with maintained or improved accuracy.

Findings

TKNs reduce computation by about an order of magnitude.

TKNs outperform baseline models on various tasks.

The method is applicable to different CNN architectures.

Abstract

We propose Attentive Regularization (AR), a method to constrain the activation maps of kernels in Convolutional Neural Networks (CNNs) to specific regions of interest (ROIs). Each kernel learns a location of specialization along with its weights through standard backpropagation. A differentiable attention mechanism requiring no additional supervision is used to optimize the ROIs. Traditional CNNs of different types and structures can be modified with this idea into equivalent Targeted Kernel Networks (TKNs), while keeping the network size nearly identical. By restricting kernel ROIs, we reduce the number of sliding convolutional operations performed throughout the network in its forward pass, speeding up both training and inference. We evaluate our proposed architecture on both synthetic and natural tasks across multiple domains. TKNs obtain significant improvements over baselines, requiring less computation (around an order of magnitude) while achieving superior performance.