Doubly Convolutional Neural Networks

TL;DR

Doubly Convolutional Neural Networks (DCNNs) enhance traditional CNNs by using filter groups that are translated versions of each other, leading to improved accuracy and reduced memory usage across multiple image classification benchmarks.

Contribution

This paper introduces DCNNs, a novel architecture that leverages filter translation groups, and demonstrates their effectiveness through extensive experiments on standard benchmarks.

Findings

DCNNs outperform traditional CNNs on CIFAR-10, CIFAR-100, and ImageNet.

Replacing layers with doubly convolutional layers improves performance.

DCNNs can reduce memory footprint without sacrificing accuracy.

Abstract

Building large models with parameter sharing accounts for most of the success of deep convolutional neural networks (CNNs). In this paper, we propose doubly convolutional neural networks (DCNNs), which significantly improve the performance of CNNs by further exploring this idea. In stead of allocating a set of convolutional filters that are independently learned, a DCNN maintains groups of filters where filters within each group are translated versions of each other. Practically, a DCNN can be easily implemented by a two-step convolution procedure, which is supported by most modern deep learning libraries. We perform extensive experiments on three image classification benchmarks: CIFAR-10, CIFAR-100 and ImageNet, and show that DCNNs consistently outperform other competing architectures. We have also verified that replacing a convolutional layer with a doubly convolutional layer at any depth of a CNN can improve its performance. Moreover, various design choices of DCNNs are demonstrated, which shows that DCNN can serve the dual purpose of building more accurate models and/or reducing the memory footprint without sacrificing the accuracy.