Low-memory convolutional neural networks through incremental depth-first processing

TL;DR

This paper presents a novel incremental depth-first processing method for CNN inference that significantly reduces memory usage, making it suitable for embedded systems with strict memory constraints.

Contribution

It introduces a depth-first updating scheme for CNN inference that bounds memory usage and is adaptable to 1D and 2D inputs, enabling low-memory neural network processing.

Findings

Memory usage is constant for 1D inputs.

Memory scales with the square root of input size for 2D inputs.

The method enables CNN inference on memory-limited embedded devices.

Abstract

We introduce an incremental processing scheme for convolutional neural network (CNN) inference, targeted at embedded applications with limited memory budgets. Instead of processing layers one by one, individual input pixels are propagated through all parts of the network they can influence under the given structural constraints. This depth-first updating scheme comes with hard bounds on the memory footprint: the memory required is constant in the case of 1D input and proportional to the square root of the input dimension in the case of 2D input.