SMM-Conv: Scalar Matrix Multiplication with Zero Packing for Accelerated Convolution

TL;DR

This paper introduces SMM-Conv, a new method for faster convolution inference on CPUs that reduces memory usage and improves speed by leveraging scalar-matrix multiplication instead of traditional im2col and GEMM approaches.

Contribution

The paper proposes a novel scalar-matrix multiplication technique that accelerates convolution inference and reduces memory overhead compared to traditional im2col and GEMM methods.

Findings

Significant speedup over existing methods in common network architectures

Reduced memory buffer requirements during inference

Improved efficiency of convolution computations on CPU architectures

Abstract

We present a novel approach for accelerating convolutions during inference for CPU-based architectures. The most common method of computation involves packing the image into the columns of a matrix (im2col) and performing general matrix multiplication (GEMM) with a matrix of weights. This results in two main drawbacks: (a) im2col requires a large memory buffer and can experience inefficient memory access, and (b) while GEMM is highly optimized for scientific matrices multiplications, it is not well suited for convolutions. We propose an approach that takes advantage of scalar-matrix multiplication and reduces memory overhead. Our experiments with commonly used network architectures demonstrate a significant speedup compared to existing indirect methods.