A Novel mapping for visual to auditory sensory substitution

TL;DR

This paper introduces a new method for converting visual environmental features into audio cues for sensory substitution, improving training efficiency and achieving high accuracy in blind object recognition.

Contribution

The study presents a novel mapping technique that enhances training efficiency and recognition accuracy over previous sinusoidal tone-based methods.

Findings

Higher training time efficiency compared to previous methods

Achieved 88.05% accuracy in blind object recognition

Effective conversion of visual features to audio cues

Abstract

visual information can be converted into audio stream via sensory substitution devices in order to give visually impaired people the chance of perception of their surrounding easily and simultaneous to performing everyday tasks. In this study, visual environmental features namely, coordinate, type of objects and their size are assigned to audio features related to music tones such as frequency, time duration and note permutations. Results demonstrated that this new method has more training time efficiency in comparison with our previous method named VBTones which sinusoidal tones were applied. Moreover, results in blind object recognition for real objects was achieved 88.05 on average.