DEX: Data Channel Extension for Efficient CNN Inference on Tiny AI Accelerators

TL;DR

This paper introduces DEX, a novel data channel extension method that enhances CNN accuracy on tiny AI accelerators by incorporating additional spatial information without increasing inference latency.

Contribution

DEX is a new approach that extends input data channels using underutilized resources, improving accuracy in TinyML models on tiny AI accelerators.

Findings

Average accuracy improvement of 3.5 percentage points

No increase in inference latency

Effective across multiple models and datasets

Abstract

Tiny machine learning (TinyML) aims to run ML models on small devices and is increasingly favored for its enhanced privacy, reduced latency, and low cost. Recently, the advent of tiny AI accelerators has revolutionized the TinyML field by significantly enhancing hardware processing power. These accelerators, equipped with multiple parallel processors and dedicated per-processor memory instances, offer substantial performance improvements over traditional microcontroller units (MCUs). However, their limited data memory often necessitates downsampling input images, resulting in accuracy degradation. To address this challenge, we propose Data channel EXtension (DEX), a novel approach for efficient CNN execution on tiny AI accelerators. DEX incorporates additional spatial information from original images into input images through patch-wise even sampling and channel-wise stacking, effectively extending data across input channels. By leveraging underutilized processors and data memory for channel extension, DEX facilitates parallel execution without increasing inference latency. Our evaluation with four models and four datasets on tiny AI accelerators demonstrates that this simple idea improves accuracy on average by 3.5%p while keeping the inference latency the same on the AI accelerator. The source code is available at https://github.com/Nokia-Bell-Labs/data-channel-extension.