SSSUMO: Real-Time Semi-Supervised Submovement Decomposition

TL;DR

SSSUMO is a semi-supervised deep learning model that accurately and efficiently decomposes submovements in real-time, overcoming previous limitations in accuracy, speed, and data labeling requirements.

Contribution

The paper introduces a novel semi-supervised deep learning framework for submovement decomposition that learns from synthetic and unlabeled data, achieving state-of-the-art performance and real-time operation.

Findings

Achieves high accuracy on synthetic and human motion datasets.

Operates in less than a millisecond per input second, enabling real-time analysis.

Outperforms existing methods, especially on challenging, noisy datasets.

Abstract

This paper introduces a SSSUMO, semi-supervised deep learning approach for submovement decomposition that achieves state-of-the-art accuracy and speed. While submovement analysis offers valuable insights into motor control, existing methods struggle with reconstruction accuracy, computational cost, and validation, due to the difficulty of obtaining hand-labeled data. We address these challenges using a semi-supervised learning framework. This framework learns from synthetic data, initially generated from minimum-jerk principles and then iteratively refined through adaptation to unlabeled human movement data. Our fully convolutional architecture with differentiable reconstruction significantly surpasses existing methods on both synthetic and diverse human motion datasets, demonstrating robustness even in high-noise conditions. Crucially, the model operates in real-time (less than a millisecond per input second), a substantial improvement over optimization-based techniques. This enhanced performance facilitates new applications in human-computer interaction, rehabilitation medicine, and motor control studies. We demonstrate the model's effectiveness across diverse human-performed tasks such as steering, rotation, pointing, object moving, handwriting, and mouse-controlled gaming, showing notable improvements particularly on challenging datasets where traditional methods largely fail. Training and benchmarking source code, along with pre-trained model weights, are made publicly available at https://github.com/dolphin-in-a-coma/sssumo.