From Vision to Assistance: Gaze and Vision-Enabled Adaptive Control for a Back-Support Exoskeleton

TL;DR

This paper introduces a vision-based control system for a back-support exoskeleton that uses egocentric vision and gaze tracking to provide context-aware assistance, improving user comfort and task fluency.

Contribution

It presents a novel real-time vision-gated control framework integrating grasp detection, FSM, and adaptive torque control for lumbar exoskeletons, enhancing responsiveness and user acceptance.

Findings

Vision-enabled assistance reduces perceived physical demand.

Users prefer the vision-gated mode for comfort and trust.

Earlier and stronger assistance observed with vision integration.

Abstract

Back-support exoskeletons have been proposed to mitigate spinal loading in industrial handling, yet their effectiveness critically depends on timely and context-aware assistance. Most existing approaches rely either on load-estimation techniques (e.g., EMG, IMU) or on vision systems that do not directly inform control. In this work, we present a vision-gated control framework for an active lumbar occupational exoskeleton that leverages egocentric vision with wearable gaze tracking. The proposed system integrates real-time grasp detection from a first-person YOLO-based perception system, a finite-state machine (FSM) for task progression, and a variable admittance controller to adapt torque delivery to both posture and object state. A user study with 15 participants performing stooping load lifting trials under three conditions (no exoskeleton, exoskeleton without vision, exoskeleton with vision) shows that vision-gated assistance significantly reduces perceived physical demand and improves fluency, trust, and comfort. Quantitative analysis reveals earlier and stronger assistance when vision is enabled, while questionnaire results confirm user preference for the vision-gated mode. These findings highlight the potential of egocentric vision to enhance the responsiveness, ergonomics, safety, and acceptance of back-support exoskeletons.