Shifting Attention to You: Personalized Brain-Inspired AI Models

TL;DR

This paper presents a personalized brain-inspired AI model that integrates human behavioral and neural data, significantly enhancing perception prediction and aligning AI computations with individual neural dynamics.

Contribution

It introduces a novel fine-tuning approach embedding human biases and neural data into CLIP, enabling personalized and interpretable AI systems.

Findings

Doubles behavioral performance over baseline CLIP.

Improves prediction of human similarity judgments.

Tracks individual neural response dynamics.

Abstract

The integration of human and artificial intelligence offers a powerful avenue for advancing our understanding of information processing, as each system provides unique computational insights. However, despite the promise of human-AI integration, current AI models are largely trained on massive datasets, optimized for population-level performance, lacking mechanisms to align their computations with individual users' perceptual semantics and neural dynamics. Here we show that integrating human behavioral insights and millisecond scale neural data within a fine tuned CLIP based model not only captures generalized and individualized aspects of perception but also over doubles behavioral performance compared to the unmodified CLIP baseline. By embedding human inductive biases and mirroring dynamic neural processes during training, personalized neural fine tuning improves predictions of human similarity judgments and tracks the temporal evolution of individual neural responses. Our work establishes a novel, interpretable framework for designing adaptive AI systems, with broad implications for neuroscience, personalized medicine, and human-computer interaction.