Position and Orientation-Aware One-Shot Learning for Medical Action Recognition from Signal Data

TL;DR

This paper introduces a position and orientation-aware one-shot learning framework for medical action recognition from signal data, utilizing signal-level image generation, cross-attention, and dynamic time warping to improve accuracy and privacy preservation.

Contribution

The work presents a novel two-stage framework that combines privacy-preserved features, cross-attention, and temporal alignment for enhanced medical action recognition from signal data.

Findings

Outperforms state-of-the-art methods on NTU RGB+D 60, NTU RGB+D 120, and PKU-MMD datasets.

Achieves 2.7%, 6.2%, and 4.1% accuracy improvements respectively.

Effectively reduces recognition bias and temporal mismatching issues.

Abstract

In this work, we propose a position and orientation-aware one-shot learning framework for medical action recognition from signal data. The proposed framework comprises two stages and each stage includes signal-level image generation (SIG), cross-attention (CsA), dynamic time warping (DTW) modules and the information fusion between the proposed privacy-preserved position and orientation features. The proposed SIG method aims to transform the raw skeleton data into privacy-preserved features for training. The CsA module is developed to guide the network in reducing medical action recognition bias and more focusing on important human body parts for each specific action, aimed at addressing similar medical action related issues. Moreover, the DTW module is employed to minimize temporal mismatching between instances and further improve model performance. Furthermore, the proposed privacy-preserved orientation-level features are utilized to assist the position-level features in both of the two stages for enhancing medical action recognition performance. Extensive experimental results on the widely-used and well-known NTU RGB+D 60, NTU RGB+D 120, and PKU-MMD datasets all demonstrate the effectiveness of the proposed method, which outperforms the other state-of-the-art methods with general dataset partitioning by 2.7%, 6.2% and 4.1%, respectively.