Static or Temporal? Semantic Scene Simplification to Aid Wayfinding in Immersive Simulations of Bionic Vision

TL;DR

This study compares two semantic scene simplification methods in immersive virtual reality to improve wayfinding for simulated bionic vision, demonstrating that both enhance performance and user experience in cluttered environments.

Contribution

It introduces and evaluates two novel semantic preprocessing approaches, SemanticEdges and SemanticRaster, for aiding navigation in prosthetic vision simulations.

Findings

Both methods improved navigation success over baseline.

SemanticEdges increased success odds; SemanticRaster reduced collisions.

Adaptive semantic preprocessing benefits low-bandwidth visual interfaces.

Abstract

Visual neuroprostheses (bionic eye) aim to restore a rudimentary form of vision by translating camera input into patterns of electrical stimulation. To improve scene understanding under extreme resolution and bandwidth constraints, prior work has explored computer vision techniques such as semantic segmentation and depth estimation. However, presenting all task-relevant information simultaneously can overwhelm users in cluttered environments. We compare two complementary approaches to semantic preprocessing in immersive virtual reality: SemanticEdges, which highlights all relevant objects at once, and SemanticRaster, which staggers object categories over time to reduce visual clutter. Using a biologically grounded simulation of prosthetic vision, 18 sighted participants performed a wayfinding task in a dynamic urban environment across three conditions: edge-based baseline (Control), SemanticEdges, and SemanticRaster. Both semantic strategies improved performance and user experience relative to the baseline, with each offering distinct trade-offs: SemanticEdges increased the odds of success, while SemanticRaster boosted the likelihood of collision-free completions. These findings underscore the value of adaptive semantic preprocessing for prosthetic vision and, more broadly, may inform the design of low-bandwidth visual interfaces in XR that must balance information density, task relevance, and perceptual clarity.