# Influence of simple crossmodal correspondence on interpretation of spoken intent

## TL;DR

This study shows that visual cues can influence how people interpret spoken language, such as distinguishing questions from statements.

## Contribution

It demonstrates that simple crossmodal correspondences can affect high-level language interpretation.

## Key findings

- Changes in visual features like elevation or lightness can bias perception of spoken intent.
- Size/pitch does not influence language judgments, as confirmed by follow-up experiments.

## Abstract

Crossmodal correspondences (CMCs) are consistent associations between sensory features from different modalities. Previous research has demonstrated that these pairings can be recruited in the resolution of the Rubin face-vase illusion. This research builds upon and expands these findings into the resolution of language ambiguity. The study uses the feature pairings of elevation/pitch, lightness/pitch, and size/pitch to influence the perceived intonation of a spoken utterance. Participants listened to a range of auditory stimuli, varying in their intent, and attempted to classify them as questions or statements. We found that change in the irrelevant visual features of elevation or lightness could bias their judgements in a manner consistent with an actual change in the pitch of the auditory stimulus. Size/pitch does not appear to affect language judgements, and we confirm this in a series of follow-up experiments. This suggests that simple crossmodal correspondences can influence high-level ambiguity resolution, at least in some cases. These findings have important implications for the use of visual cues in audiovisual language studies such as the visual world paradigm.

The online version contains supplementary material available at 10.3758/s13414-025-03129-z.