Multivariate Gaussian Representation Learning for Medical Action Evaluation

TL;DR

This paper introduces a new benchmark dataset and a Gaussian-based framework for fine-grained medical action evaluation, improving accuracy and robustness in rapid, complex motions.

Contribution

The paper presents CPREval-6k, a comprehensive medical action dataset, and GaussMedAct, a novel Gaussian encoding method for enhanced spatiotemporal modeling in medical motion analysis.

Findings

Achieves 92.1% Top-1 accuracy with real-time inference.

Outperforms baseline by +5.9% accuracy with reduced FLOPs.

Demonstrates robustness across multiple datasets.

Abstract

Fine-grained action evaluation in medical vision faces unique challenges due to the unavailability of comprehensive datasets, stringent precision requirements, and insufficient spatiotemporal dynamic modeling of very rapid actions. To support development and evaluation, we introduce CPREval-6k, a multi-view, multi-label medical action benchmark containing 6,372 expert-annotated videos with 22 clinical labels. Using this dataset, we present GaussMedAct, a multivariate Gaussian encoding framework, to advance medical motion analysis through adaptive spatiotemporal representation learning. Multivariate Gaussian Representation projects the joint motions to a temporally scaled multi-dimensional space, and decomposes actions into adaptive 3D Gaussians that serve as tokens. These tokens preserve motion semantics through anisotropic covariance modeling while maintaining robustness to spatiotemporal noise. Hybrid Spatial Encoding, employing a Cartesian and Vector dual-stream strategy, effectively utilizes skeletal information in the form of joint and bone features. The proposed method achieves 92.1% Top-1 accuracy with real-time inference on the benchmark, outperforming baseline by +5.9% accuracy with only 10% FLOPs. Cross-dataset experiments confirm the superiority of our method in robustness.