Simple Models, Rich Representations: Visual Decoding from Primate Intracortical Neural Signals

TL;DR

This study demonstrates that simple models focusing on temporal dynamics can effectively decode visual information from primate neural signals, leading to high accuracy and plausible image generation, informing brain-computer interface development.

Contribution

The paper shows that temporal modeling is key for neural decoding and introduces a modular pipeline for generating images from brain activity using simple models.

Findings

Simple models with temporal attention achieve 70% top-1 accuracy.

Decoding performance is mainly influenced by temporal dynamics rather than model complexity.

A generative pipeline can produce plausible images from 200 ms of neural data.

Abstract

Understanding how neural activity gives rise to perception is a central challenge in neuroscience. We address the problem of decoding visual information from high-density intracortical recordings in primates, using the THINGS Ventral Stream Spiking Dataset. We systematically evaluate the effects of model architecture, training objectives, and data scaling on decoding performance. Results show that decoding accuracy is mainly driven by modeling temporal dynamics in neural signals, rather than architectural complexity. A simple model combining temporal attention with a shallow MLP achieves up to 70% top-1 image retrieval accuracy, outperforming linear baselines as well as recurrent and convolutional approaches. Scaling analyses reveal predictable diminishing returns with increasing input dimensionality and dataset size. Building on these findings, we design a modular generative decoding pipeline that combines low-resolution latent reconstruction with semantically conditioned diffusion, generating plausible images from 200 ms of brain activity. This framework provides principles for brain-computer interfaces and semantic neural decoding.