Parameter-Efficient Deep Learning for Ultrasound-Based Human-Machine Interfaces

TL;DR

This paper systematically compares deep learning models for ultrasound-based hand pose estimation, demonstrating that optimized model, input, and training strategies significantly improve performance while reducing complexity.

Contribution

It provides a comprehensive benchmark of six deep learning models on the Ultra-Pro dataset and introduces a novel optimized model with fewer parameters and higher accuracy.

Findings

The 4-layer UDACNN outperforms XceptionTime by 2.28 percentage points.

Using RF signal envelope as input improves model performance.

Optimized combination of model, preprocessing, and training enhances HMI accuracy.

Abstract

Ultrasound (US) has emerged as a promising modality for Human-Machine Interfaces (HMIs), with recent research efforts exploring its potential for Hand Pose Estimation (HPE). A reliable solution to this problem could introduce interfaces with simultaneous support for up to 23 degrees of freedom encompassing all hand and wrist kinematics, thereby allowing far richer and more intuitive interaction strategies. Despite these promising results, a systematic comparison of models, input modalities and training strategies is missing from the literature. Moreover, there is only one publicly available dataset, namely the Ultrasound Adaptive Prosthetic Control (Ultra-Pro) dataset, enabling reproducible benchmarking and iterative model development. In this paper, we compare the performance of six different deep learning models, selected based on diverse criteria, on this benchmark. We demonstrate that, by using a step learning rate scheduler and the envelope of the RF signals as input modality, our 4-layer deep UDACNN surpasses XceptionTime's performance by $2.28$ percentage points while featuring $87.52\%$ fewer parameters. This result ($77.72\%$) constitutes an absolute improvement of $0.88\%$ from previously reported baselines. According to our findings, the appropriate combination of model, preprocessing and training algorithm is crucial for optimizing HMI performance.