Adaptive Test-Time Augmentation for Low-Power CPU

TL;DR

This paper introduces AdapTTA, an adaptive test-time augmentation method for low-power CPUs that dynamically adjusts the number of inference passes, significantly reducing latency while maintaining accuracy improvements.

Contribution

The paper presents AdapTTA, a novel adaptive TTA approach that reduces latency on embedded CPUs by dynamically controlling inference passes based on input complexity.

Findings

Latency savings from 1.49X to 2.21X compared to static TTA

Maintains the same accuracy gains as traditional TTA

Effective deployment on ARM Cortex-A CPUs for image classification

Abstract

Convolutional Neural Networks (ConvNets) are trained offline using the few available data and may therefore suffer from substantial accuracy loss when ported on the field, where unseen input patterns received under unpredictable external conditions can mislead the model. Test-Time Augmentation (TTA) techniques aim to alleviate such common side effect at inference-time, first running multiple feed-forward passes on a set of altered versions of the same input sample, and then computing the main outcome through a consensus of the aggregated predictions. Unfortunately, the implementation of TTA on embedded CPUs introduces latency penalties that limit its adoption on edge applications. To tackle this issue, we propose AdapTTA, an adaptive implementation of TTA that controls the number of feed-forward passes dynamically, depending on the complexity of the input. Experimental results on state-of-the-art ConvNets for image classification deployed on a commercial ARM Cortex-A CPU demonstrate AdapTTA reaches remarkable latency savings, from 1.49X to 2.21X, and hence a higher frame rate compared to static TTA, still preserving the same accuracy gain.