A Constant-Time Implementation Methodology for Activation Functions on Microcontrollers

TL;DR

This paper introduces a methodology for implementing activation functions on microcontrollers that ensures constant execution time, enhancing resistance to timing side-channel attacks in embedded neural network inference.

Contribution

It presents a novel, practical approach combining branchless selection, fixed-cost approximation, and cycle alignment for secure activation functions on embedded devices.

Findings

Achieves identical cycle counts for all inputs, e.g., 88 cycles for three functions.

Maintains high numerical accuracy of approximated nonlinear functions.

Demonstrates vulnerability of desynchronization countermeasure to timing attacks.

Abstract

Embedded neural-network inference can leak information through timing side channels, including leakage caused by the evaluation of activation functions. This work proposes a constant-time implementation methodology for activation functions on embedded microcontrollers and validates it on ReLU, sigmoid, tanh, GELU, and Swish on an ARM Cortex-M4 platform. The proposed methodology combines branchless selection, fixed-cost Pad\'e-based approximation, dummy arithmetic where needed, and cycle alignment to obtain timing-regular activation-function implementations. As motivation, we also evaluate a desynchronization-based countermeasure and show that it remains vulnerable to a template-based timing attack. Experimental results show that the resulting protected implementations achieve identical cycle counts for all tested inputs, including (88) cycles in the three-function setting and (108) cycles in the five-function setting. At the same time, the numerical-error analysis indicates that the approximated nonlinear functions retain high accuracy. These results suggest that the proposed methodology provides a practical basis for constructing side-channel-resistant activation functions in embedded inference.