Why early tactile speech aids may have failed: no perceptual integration of tactile and auditory signals

TL;DR

This study investigates why tactile speech aids failed by testing whether tactile and auditory signals are perceptually integrated or merely cognitively associated, finding evidence for the latter and explaining past failures.

Contribution

The paper provides experimental evidence that tactile and auditory integration in speech aids is cognitive, not perceptual, clarifying why early tactile speech aids were unsuccessful.

Findings

Tactile cues influence auditory discrimination only above tactile threshold.

No McGurk-like effect was observed with tactile stimuli.

Tactile and auditory integration appears to be cognitive, not perceptual.

Abstract

Tactile speech aids, though extensively studied in the 1980s and 90s, never became a commercial success. A hypothesis to explain this failure might be that it is difficult to obtain true perceptual integration of a tactile signal with information from auditory speech: exploitation of tactile cues from a tactile aid might require cognitive effort and so prevent speech understanding at the high rates typical of everyday speech. To test this hypothesis, we attempted to create true perceptual integration of tactile with auditory information in what might be considered the simplest situation encountered by a hearing-impaired listener. We created an auditory continuum between the syllables BA and VA, and trained participants to associate BA to one tactile stimulus VA to another tactile stimulus. After training, we tested if auditory discrimination along the continuum between the two syllables could be biased by incongruent tactile stimulation. We found that such a bias occurred only when the tactile stimulus was above its previously measured tactile discrimination threshold. Such a pattern is compatible with the idea that the effect is due to a cognitive or decisional strategy, rather than to truly perceptual integration. We therefore ran a further study, where we created a tactile version of the McGurk effect. We extensively trained two Subjects over six days to associate four recorded auditory syllables with four corresponding apparent motion tactile patterns. In a subsequent test, we presented stimulation that was either congruent or incongruent with the learnt association, and asked Subjects to report the syllable they perceived. We found no analog to the McGurk effect. These findings strengthen our hypothesis according to which tactile aids failed because integration of tactile cues with auditory speech occurred at a cognitive or decisional level, rather than truly at a perceptual level