Explore, Listen, Inspect: Supporting Multimodal Interaction with 3D Surface and Point Data Visualizations

TL;DR

This paper presents DIXTRAL, a web-based system that enables blind and low-vision users to interact with 3D surface and point data visualizations through multimodal feedback and accessible navigation methods.

Contribution

It introduces a novel multimodal interaction system co-designed with BLV researchers, enhancing non-visual access to 3D visualizations in browser environments.

Findings

Synchronized auditory, visual, and textual feedback improves spatial understanding.

Keyboard and gamepad navigation support enhances exploration and orientation.

Iterative testing with BLV users demonstrates usability and effectiveness.

Abstract

Blind and low-vision (BLV) users remain largely excluded from three-dimensional (3D) surface and point data visualizations due to the reliance on visual interaction. Existing approaches inadequately support non-visual access, especially in browser-based environments. This study introduces DIXTRAL, a hosted web-native system, co-designed with BLV researchers to address these gaps through multimodal interaction. Conducted with two blind and one sighted researcher, this study took place over sustained design sessions. Data were gathered through iterative testing of the prototype, collecting feedback on spatial navigation, sonification, and usability. Co-design observations demonstrate that synchronized auditory, visual, and textual feedback, combined with keyboard and gamepad navigation, enhances both structure discovery and orientation. DIXTRAL aims to improve access to 3D continuous scalar fields for BLV users and inform best practices for creating inclusive 3D visualizations.