Is Information Density Uniform when Utterances are Grounded on Perception and Discourse?

TL;DR

This study investigates how visual grounding affects the distribution of information in language, showing that multimodal context leads to more uniform information flow across diverse languages and discourse types.

Contribution

It is the first computational analysis of UID in visually grounded settings, demonstrating increased information uniformity with perceptual grounding across multiple languages.

Findings

Grounding on perception increases information uniformity.

Visual and discourse context further reduce surprisal at discourse onsets.

Grounded language shows greater information flow consistency.

Abstract

The Uniform Information Density (UID) hypothesis posits that speakers are subject to a communicative pressure to distribute information evenly within utterances, minimising surprisal variance. While this hypothesis has been tested empirically, prior studies are limited exclusively to text-only inputs, abstracting away from the perceptual context in which utterances are produced. In this work, we present the first computational study of UID in visually grounded settings. We estimate surprisal using multilingual vision-and-language models over image-caption data in 30 languages and visual storytelling data in 13 languages, together spanning 11 families. We find that grounding on perception consistently smooths the distribution of information, increasing both global and local uniformity across typologically diverse languages compared to text-only settings. In visual narratives, grounding in both image and discourse contexts has additional effects, with the strongest surprisal reductions occurring at the onset of discourse units. Overall, this study takes a first step towards modelling the temporal dynamics of information flow in ecologically plausible, multimodal language use, and finds that grounded language exhibits greater information uniformity, supporting a context-sensitive formulation of UID.