Convolutional Networks with Oriented 1D Kernels

TL;DR

This paper demonstrates that a convolutional neural network composed solely of oriented 1D convolutions can match the performance of traditional 2D ConvNets on ImageNet, with optimized CUDA implementation for efficiency.

Contribution

It introduces the concept of oriented 1D kernels for ConvNets, showing they can replace 2D convolutions and improve accuracy with minimal computational overhead.

Findings

1D oriented kernels can match 2D convolution performance on ImageNet.

Custom CUDA implementation achieves near-theoretical speedup for arbitrary angles.

Oriented 1D convolutions can augment existing architectures for better accuracy.

Abstract

In computer vision, 2D convolution is arguably the most important operation performed by a ConvNet. Unsurprisingly, it has been the focus of intense software and hardware optimization and enjoys highly efficient implementations. In this work, we ask an intriguing question: can we make a ConvNet work without 2D convolutions? Surprisingly, we find that the answer is yes -- we show that a ConvNet consisting entirely of 1D convolutions can do just as well as 2D on ImageNet classification. Specifically, we find that one key ingredient to a high-performing 1D ConvNet is oriented 1D kernels: 1D kernels that are oriented not just horizontally or vertically, but also at other angles. Our experiments show that oriented 1D convolutions can not only replace 2D convolutions but also augment existing architectures with large kernels, leading to improved accuracy with minimal FLOPs increase. A key contribution of this work is a highly-optimized custom CUDA implementation of oriented 1D kernels, specialized to the depthwise convolution setting. Our benchmarks demonstrate that our custom CUDA implementation almost perfectly realizes the theoretical advantage of 1D convolution: it is faster than a native horizontal convolution for any arbitrary angle. Code is available at https://github.com/princeton-vl/Oriented1D.