# Coupling Causal Inference and Cross-Modal Recalibration: A Unified Framework for Adaptive Multisensory Perception

## TL;DR

This paper explores how the brain decides to combine or separate sensory information and adapts when senses conflict, using a framework that links causal inference with recalibration.

## Contribution

The paper introduces a unified framework linking causal inference and recalibration in multisensory perception.

## Key findings

- Causal inference determines whether sensory signals should be integrated or segregated.
- Recalibration reshapes sensory representations and expectations, influencing future causal judgments.

## Abstract

We experience the world by combining information from sight, hearing, touch, and balance. This usually helps us perceive more accurately, but the senses are noisy and sometimes describe different events. The brain therefore faces two linked challenges. First, it must decide, moment by moment, whether signals from different senses come from the same source and should be combined, or from different sources and should be kept separate. Second, when a mismatch between senses happens repeatedly, the brain can gradually “re-tune” how it interprets one or more senses so that the conflict becomes smaller in the future. This review explains evidence for both processes across several everyday situations, such as judging where a sound comes from when sight and sound disagree, feeling that a fake hand is part of your body when sight and touch match, and estimating self-motion when visual and vestibular cues conflict. We argue that the decision about “same source or different sources” likely helps control when recalibration happens, while recalibration can also reshape future decisions. Understanding this link may improve approaches to training, rehabilitation, and virtual reality design.

Perception in natural environments relies on the integration of information from multiple sensory modalities. Although combining sensory cues can improve perceptual accuracy, sensory signals are inherently noisy and do not always arise from a common source. As a result, the brain must continuously determine whether multisensory signals should be integrated or segregated, while also adapting sensory representations when discrepancies persist over time. Multisensory causal inference provides a principled framework of how the brain infers the causal structure underlying sensory inputs and flexibly arbitrates between integration and segregation. In parallel, extensive behavioral work has shown that prolonged exposure to cross-modal conflicts induces multisensory recalibration, leading to persistent changes in unisensory and multisensory perception. Despite progress in both areas, the relationship between causal inference and recalibration has remained unclear. In this review, we synthesize behavioral, computational, and neurophysiological evidence to argue that causal inference and recalibration form a coupled adaptive system operating across distinct time scales. We propose that causal inference constrains whether sensory discrepancies drive recalibration, whereas recalibration reshapes sensory representations and expectations, thereby influencing future causal judgments.