Breast-Rehab: A Postoperative Breast Cancer Rehabilitation Training Assessment System Based on Human Action Recognition

TL;DR

Breast-Rehab is a low-cost, AI-powered system that uses human action recognition and language models to monitor and assess breast cancer patients' at-home upper limb rehabilitation exercises, improving accessibility and clinical guidance.

Contribution

The paper introduces a novel integrated system combining human action recognition with a retrieval-augmented generation framework for effective at-home rehabilitation assessment.

Findings

Outperforms standard models in exercise video segmentation.

Achieved an average of 0.59 exercise sessions per day in clinical validation.

Demonstrated system feasibility and user acceptance in a preliminary study.

Abstract

Postoperative upper limb dysfunction is prevalent among breast cancer survivors, yet their adherence to at-home rehabilitation exercises is low amidst limited nursing resources. The hardware overhead of commonly adopted VR-based mHealth solutions further hinders their widespread clinical application. Therefore, we developed Breast-Rehab, a novel, low-cost mHealth system to provide patients with out-of-hospital upper limb rehabilitation management. Breast-Rehab integrates a bespoke human action recognition algorithm with a retrieval-augmented generation (RAG) framework. By fusing visual and 3D skeletal data, our model accurately segments exercise videos recorded in uncontrolled home environments, outperforming standard models. These segmented clips, combined with a domain-specific knowledge base, guide a multi-modal large language model to generate clinically relevant assessment reports. This approach significantly reduces computational overhead and mitigates model hallucinations. We implemented the system as a WeChat Mini Program and a nurse-facing dashboard. A preliminary clinical study validated the system's feasibility and user acceptance, with patients achieving an average exercise frequency of 0.59 sessions/day over a two-week period. This work thus presents a complete, validated pipeline for AI-driven, at-home rehabilitation monitoring.