Decoupled Dynamic Filter Networks

TL;DR

This paper introduces Decoupled Dynamic Filter (DDF), a novel convolution method that reduces computational costs and enhances performance by decoupling spatial and channel filters, outperforming standard and depth-wise convolutions.

Contribution

The paper proposes DDF, a decoupled dynamic filter that improves CNN efficiency and accuracy by separating spatial and channel filtering, inspired by attention mechanisms.

Findings

ResNet50/101 accuracy improved by 1.9%/1.3%

Computational costs nearly halved

DDF outperforms standard and content-adaptive layers in detection and upsampling tasks

Abstract

Convolution is one of the basic building blocks of CNN architectures. Despite its common use, standard convolution has two main shortcomings: Content-agnostic and Computation-heavy. Dynamic filters are content-adaptive, while further increasing the computational overhead. Depth-wise convolution is a lightweight variant, but it usually leads to a drop in CNN performance or requires a larger number of channels. In this work, we propose the Decoupled Dynamic Filter (DDF) that can simultaneously tackle both of these shortcomings. Inspired by recent advances in attention, DDF decouples a depth-wise dynamic filter into spatial and channel dynamic filters. This decomposition considerably reduces the number of parameters and limits computational costs to the same level as depth-wise convolution. Meanwhile, we observe a significant boost in performance when replacing standard convolution with DDF in classification networks. ResNet50 / 101 get improved by 1.9% and 1.3% on the top-1 accuracy, while their computational costs are reduced by nearly half. Experiments on the detection and joint upsampling networks also demonstrate the superior performance of the DDF upsampling variant (DDF-Up) in comparison with standard convolution and specialized content-adaptive layers.