Vision-based Wearable Steering Assistance for People with Impaired Vision in Jogging

TL;DR

This paper presents a vision-based wearable steering aid for visually impaired joggers, utilizing a lightweight multitask network and a new dataset to improve safety and speed in outdoor athletics environments.

Contribution

It introduces a novel multitask detection network and a new annotated dataset for athletics tracks, enhancing perception and planning for visually impaired jogging assistance.

Findings

Achieved 1.34 m/s jogging speed with assistance

Developed a lightweight multitask network for track and obstacle detection

Demonstrated system adaptability in outdoor sports scenarios

Abstract

Outdoor sports pose a challenge for people with impaired vision. The demand for higher-speed mobility inspired us to develop a vision-based wearable steering assistance. To ensure broad applicability, we focused on a representative sports environment, the athletics track. Our efforts centered on improving the speed and accuracy of perception, enhancing planning adaptability for the real world, and providing swift and safe assistance for people with impaired vision. In perception, we engineered a lightweight multitask network capable of simultaneously detecting track lines and obstacles. Additionally, due to the limitations of existing datasets for supporting multi-task detection in athletics tracks, we diligently collected and annotated a new dataset (MAT) containing 1000 images. In planning, we integrated the methods of sampling and spline curves, addressing the planning challenges of curves. Meanwhile, we utilized the positions of the track lines and obstacles as constraints to guide people with impaired vision safely along the current track. Our system is deployed on an embedded device, Jetson Orin NX. Through outdoor experiments, it demonstrated adaptability in different sports scenarios, assisting users in achieving free movement of 400-meter at an average speed of 1.34 m/s, meeting the level of normal people in jogging. Our MAT dataset is publicly available from https://github.com/snoopy-l/MAT