Modeling Visual Hallucination: A Generative Adversarial Network Framework

TL;DR

This paper introduces Hallu-GAN, a generative adversarial network framework inspired by neurobiological mechanisms, to model and analyze visual hallucinations and their neural interactions, especially in disorders like Charles Bonnet syndrome.

Contribution

The study proposes a novel GAN-based model that simulates visual hallucinations by incorporating neurobiological insights and EEG data, extending understanding of hallucination mechanisms.

Findings

Disturbances in the ventral stream can induce hallucinations.

The Hallu-GAN+ model effectively simulates hallucinating and non-hallucinating states.

EEG data integration enhances analysis of hallucination phenomena.

Abstract

Visual hallucination refers to the perception of recognizable things that are not present. These phenomena are commonly linked to a range of neurological/psychiatric disorders. Despite ongoing research, the mechanisms through which the visual system generates hallucinations from real-world environments are still not well understood. Abnormal interactions between different regions of the brain responsible for perception are known to contribute to the occurrence of visual hallucinations. In this study, we propose and extend a generative neural network-based framework to address challenges within the visual system, aiming to create goal-driven models inspired by neurobiological mechanisms of visual hallucinations. We focus on the adversarial interactions between the visual system and the frontal lobe regions, proposing the Hallu-GAN model to suggest how these interactions can give rise to visual hallucinations. The architecture of the Hallu-GAN model is based on generative adversarial networks. Our simulation results indicate that disturbances in the ventral stream can lead to visual hallucinations. To further analyze the impact of other brain regions on the visual system, we extend the Hallu-GAN model by adding EEG data from individuals. This extended model, referred to as Hallu-GAN+, enables the examination of both hallucinating and non-hallucinating states. By training the Hallu-GAN+ model with EEG data from an individual with Charles Bonnet syndrome, we demonstrated its utility in analyzing the behavior of those experiencing hallucinations. Our simulation results confirmed the capability of the proposed model in resembling the visual system in both healthy and hallucinating states.