# Diffusion models enable zero-shot pose estimation for lower-limb prosthetic users

## TL;DR

This paper introduces a new method using diffusion models to improve gait analysis for lower-limb prosthetic users by generating synthetic images that standard pose estimation tools can process.

## Contribution

The novel zero-shot approach uses diffusion models to transform prosthetic limb images into able-bodied representations without retraining or additional data.

## Key findings

- The method reduced keypoint coordinate errors by 37% for transtibial and 76% for transfemoral prosthetic limbs.
- It enabled the detection of gait deviations like reduced knee flexion and altered limb kinematics.
- The approach works with existing pose estimation models and consumer cameras, offering potential for personalized rehabilitation.

## Abstract

Quantitative gait analysis is important for assessing and rehabilitating lower-limb prosthetic users, but markerless motion capture has been challenging for this population due to the difficulty in detecting prosthetic joints using models trained primarily on able-bodied individuals. This study proposes a zero-shot method leveraging generative diffusion models to transform prosthetic limb images into able-bodied representations that standard pose estimation models can detect, eliminating the need for additional data collection or model retraining. Videos of unilateral transfemoral and transtibial amputees walking were obtained publicly from YouTube. For each video frame, an edge map was generated and used as input to a ControlNet diffusion model, generating a synthetic image resembling an able-bodied person while preserving the person’s original pose. These synthetic images were then passed through OpenPose. The zero-shot approach achieved substantial reductions in keypoint coordinate errors of 37% for transtibial and 76% for transfemoral prosthetic limbs compared to OpenPose on the original videos. The method enabled the identification and quantification of key gait deviations such as reduced knee flexion and altered kinematics timing between prosthetic and intact limbs. While the results demonstrate the feasibility of markerless gait analysis for lower-limb prosthetic users, the study’s findings are based on a limited dataset of publicly available videos, and caution should be exercised in generalizing the results to broader populations due to the varying nature of prosthetic designs. Nonetheless, this approach has the potential to facilitate personalized rehabilitation using standard consumer cameras and existing pose estimation models.

The application of 2D markerless gait analysis has garnered increasing interest and application within clinical settings. However, its effectiveness in the realm of lower-limb amputees has remained less than optimal due to a lack of large image datasets of lower-limb prosthetic users to train models with. In response, we introduce an innovative zero-shot method employing image generation diffusion models to achieve markerless pose estimation for lower-limb prosthetic users without the need for additional data collection and training, presenting a promising solution to gait analysis for this specific population. Our approach demonstrates an enhancement in the detection of key points on prosthetic limbs over existing methods, and enables clinicians to gain invaluable insights into the kinematics of lower-limb amputees across the gait cycle. The outcomes obtained not only serve as a proof-of-concept for the feasibility of this zero-shot approach but also underscore its potential in advancing rehabilitation through gait analysis for this unique population.