SPECTRA: An Efficient Spectral-Informed Neural Network for Sensor-Based Activity Recognition

TL;DR

SPECTRA is a spectral-temporal neural network architecture designed for real-time sensor-based activity recognition, optimizing for low latency, privacy, and resource efficiency on edge devices.

Contribution

It introduces a spectral-temporal architecture combining STFT, depthwise convolutions, and self-attention, tailored for deployment on resource-constrained edge devices.

Findings

SPECTRA matches or exceeds larger models' accuracy on HAR datasets.

It significantly reduces parameters, latency, and energy consumption.

Successfully deployed on smartphone and microcontroller for real-time HAR.

Abstract

Real time sensor based applications in pervasive computing require edge deployable models to ensure low latency privacy and efficient interaction. A prime example is sensor based human activity recognition where models must balance accuracy with stringent resource constraints. Yet many deep learning approaches treat temporal sensor signals as black box sequences overlooking spectral temporal structure while demanding excessive computation. We present SPECTRA a deployment first co designed spectral temporal architecture that integrates short time Fourier transform STFT feature extraction depthwise separable convolutions and channel wise self attention to capture spectral temporal dependencies under real edge runtime and memory constraints. A compact bidirectional GRU with attention pooling summarizes within window dynamics at low cost reducing downstream model burden while preserving accuracy. Across five public HAR datasets SPECTRA matches or approaches larger CNN LSTM and Transformer baselines while substantially reducing parameters latency and energy. Deployments on a Google Pixel 9 smartphone and an STM32L4 microcontroller further demonstrate end to end deployable realtime private and efficient HAR.