SemanticNN: Compressive and Error-Resilient Semantic Offloading for Extremely Weak Devices

TL;DR

SemanticNN is a novel semantic codec designed for weak IoT devices that enables error-resilient, efficient offloading of AI inference tasks by tolerating bit errors and adapting to dynamic network conditions, significantly reducing data transmission.

Contribution

It introduces a semantic-level error-resilient codec with adaptive decoding, a new training strategy, and asymmetry compensation to improve offloading efficiency on resource-constrained devices.

Findings

Reduces feature transmission volume by up to 344.83x.

Maintains high inference accuracy under varying error rates.

Demonstrates effectiveness on STM32 with multiple models and datasets.

Abstract

With the rapid growth of the Internet of Things (IoT), integrating artificial intelligence (AI) on extremely weak embedded devices has garnered significant attention, enabling improved real-time performance and enhanced data privacy. However, the resource limitations of such devices and unreliable network conditions necessitate error-resilient device-edge collaboration systems. Traditional approaches focus on bit-level transmission correctness, which can be inefficient under dynamic channel conditions. In contrast, we propose SemanticNN, a semantic codec that tolerates bit-level errors in pursuit of semantic-level correctness, enabling compressive and resilient collaborative inference offloading under strict computational and communication constraints. It incorporates a Bit Error Rate (BER)-aware decoder that adapts to dynamic channel conditions and a Soft Quantization (SQ)-based encoder to learn compact representations. Building on this architecture, we introduce Feature-augmentation Learning, a novel training strategy that enhances offloading efficiency. To address encoder-decoder capability mismatches from asymmetric resources, we propose XAI-based Asymmetry Compensation to enhance decoding semantic fidelity. We conduct extensive experiments on STM32 using three models and six datasets across image classification and object detection tasks. Experimental results demonstrate that, under varying transmission error rates, SemanticNN significantly reduces feature transmission volume by 56.82-344.83x while maintaining superior inference accuracy.