Gait-Based Hand Load Estimation via Deep Latent Variable Models with Auxiliary Information

TL;DR

This paper introduces a deep learning framework that estimates hand load during manual handling by fusing loaded and unloaded gait data, improving accuracy without requiring carrying style labels during deployment.

Contribution

It presents a novel deep latent variable model with cross-attention that effectively incorporates auxiliary gait information for load estimation, enhancing generalization and accuracy.

Findings

Incorporating auxiliary information improves load estimation accuracy.

Explicit fusion mechanisms outperform naive feature concatenation.

Model achieves substantial accuracy gains on real-world wearable sensor data.

Abstract

Machine learning methods are increasingly applied to ergonomic risk assessment in manual material handling, particularly for estimating carried load from gait motion data collected from wearable sensors. However, existing approaches often rely on direct mappings from loaded gait to hand load, limiting generalization and predictive accuracy. In this study, we propose an enhanced load estimation framework that incorporates auxiliary information, including baseline gait patterns during unloaded walking and carrying style. While baseline gait can be automatically captured by wearable sensors and is thus readily available at inference time, carrying style typically requires manual labeling and is often unavailable during deployment. Our model integrates deep latent variable modeling with temporal convolutional networks and bi-directional cross-attention to capture gait dynamics and fuse loaded and unloaded gait patterns. Guided by domain knowledge, the model is designed to estimate load magnitude conditioned on carrying style, while eliminating the need for carrying style labels at inference time. Experiments using real-world data collected from inertial measurement units attached to participants demonstrate substantial accuracy gains from incorporating auxiliary information and highlight the importance of explicit fusion mechanisms over naive feature concatenation.