VW-SDK: Efficient Convolutional Weight Mapping Using Variable Windows for Processing-In-Memory Architectures

TL;DR

This paper introduces VW-SDK, a novel adaptive mapping algorithm for PIM-based CNN inference that optimizes the shape of the parallel window to minimize computation cycles, significantly improving inference speed.

Contribution

The paper proposes VW-SDK, an adaptive variable-window SDK mapping algorithm that enhances PIM array utilization and reduces computation cycles in CNN inference.

Findings

VW-SDK reduces inference cycles by 1.69x on Resnet-18.

Adaptive window shaping improves PIM efficiency.

Simulation results demonstrate significant speedup.

Abstract

With their high energy efficiency, processing-in-memory (PIM) arrays are increasingly used for convolutional neural network (CNN) inference. In PIM-based CNN inference, the computational latency and energy are dependent on how the CNN weights are mapped to the PIM array. A recent study proposed shifted and duplicated kernel (SDK) mapping that reuses the input feature maps with a unit of a parallel window, which is convolved with duplicated kernels to obtain multiple output elements in parallel. However, the existing SDK-based mapping algorithm does not always result in the minimum computing cycles because it only maps a square-shaped parallel window with the entire channels. In this paper, we introduce a novel mapping algorithm called variable-window SDK (VW-SDK), which adaptively determines the shape of the parallel window that leads to the minimum computing cycles for a given convolutional layer and PIM array. By allowing rectangular-shaped windows with partial channels, VW-SDK utilizes the PIM array more efficiently, thereby further reduces the number of computing cycles. The simulation with a 512x512 PIM array and Resnet-18 shows that VW-SDK improves the inference speed by 1.69x compared to the existing SDK-based algorithm.