NeuralSet: A High-Performing Python Package for Neuro-AI

TL;DR

NeuralSet is a Python package that unifies processing of diverse neural data and stimuli, enabling scalable, efficient neuro-AI research with full provenance and compatibility with deep learning workflows.

Contribution

It introduces NeuralSet, a framework that integrates various neural recordings and stimuli processing into a scalable, memory-efficient, PyTorch-compatible system for neuro-AI research.

Findings

Supports multiple neural data modalities including fMRI, M/EEG, and spikes.

Provides a unified, scalable interface compatible with deep learning workflows.

Eliminates manual data wrangling and ensures full computational provenance.

Abstract

Artificial intelligence (AI) is increasingly central to understanding how the brain processes information. However, the integration of neuroscience and modern AI is bottlenecked by a fragmented software ecosystem. Current tools are siloed by recording modality and optimized for small-scale, in-memory workflows, limiting the use of massive, naturalistic datasets. Here, we introduce NeuralSet, a Python framework that efficiently unifies the processing of diverse neural recordings (including fMRI, M/EEG, and spikes) and complex experimental stimuli (such as text, audio, and video). By decoupling experimental metadata from lazy, memory-efficient data extraction, NeuralSet harmonizes standard neuroscientific preprocessing pipelines with pretrained deep learning embeddings. This approach provides a single PyTorch-ready interface that scales seamlessly from local prototyping to high-performance cluster execution. By eliminating manual data wrangling and ensuring full computational provenance, NeuralSet establishes a scalable, unified infrastructure for the next generation of neuro-AI research.