Listen to Your Map: An Online Representation for Spatial Sonification
Lan Wu, Craig Jin, Monisha Mushtary Uttsha, Teresa Vidal-Calleja

TL;DR
This paper presents an online spatial sonification method that converts 3D scene geometry into auditory signals to aid navigation for visually impaired individuals, using depth sensors and a novel scene representation.
Contribution
It introduces a new mapping representation for scene geometry that enables accurate, online spatial sonification aligned with human auditory perception, improving navigation aid technology.
Findings
Enhanced accuracy and coverage in spatial sonification.
Effective handling of dynamic objects in the environment.
Improved timing and suitability for real-time navigation aid.
Abstract
Robotic perception is becoming a key technology for navigation aids, especially helping individuals with visual impairments through spatial sonification. This paper introduces a mapping representation that accurately captures scene geometry for sonification, turning physical spaces into auditory experiences. Using depth sensors, we encode an incrementally built 3D scene into a compact 360-degree representation with angular and distance information, aligning this way with human auditory spatial perception. The proposed framework performs localisation and mapping via VDB-Gaussian Process Distance Fields for efficient online scene reconstruction. The key aspect is a sensor-centric structure that maintains either a 2D-circular or 3D-cylindrical raster-based projection. This spatial representation is then converted into binaural auditory signals using simple pre-recorded responses from a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTactile and Sensory Interactions · Virtual Reality Applications and Impacts · Teleoperation and Haptic Systems
