Voluntary control of semantic neural representations by imagery with conflicting visual stimulation

TL;DR

This study demonstrates that mental imagery can modulate neural representations during conflicting visual perception, showing that imagined categories influence neural activity even when perceiving different images.

Contribution

We developed a closed-loop system that infers semantic neural representations from electrocorticograms, revealing how imagery influences perception at the neural level.

Findings

Imagery can bias neural representations towards imagined categories.

Semantic vectors inferred from neural data become closer to imagined categories.

Modulation depends on category and timing of imagery.

Abstract

Neural representations of visual perception are affected by mental imagery and attention. Although attention is known to modulate neural representations, it is unknown how imagery changes neural representations when imagined and perceived images semantically conflict. We hypothesized that imagining an image would activate a neural representation during its perception even while watching a conflicting image. To test this hypothesis, we developed a closed-loop system to show images inferred from electrocorticograms using a visual semantic space. The successful control of the feedback images demonstrated that the semantic vector inferred from electrocorticograms became closer to the vector of the imagined category, even while watching images from different categories. Moreover, modulation of the inferred vectors by mental imagery depended on both the image category and time from the initiation of imagery. The closed-loop control of the semantic vectors revealed an asymmetrical interaction between visual perception and imagery.