A Transformer-based Neural Language Model that Synthesizes Brain Activation Maps from Free-Form Text Queries

TL;DR

Text2Brain is a transformer-based tool that synthesizes brain activation maps from free-form text queries, enabling efficient exploration of neuroimaging literature and hypothesis generation.

Contribution

It introduces a novel neural network model combining a transformer text encoder with a 3D image generator trained on extensive neuroimaging data.

Findings

Successfully synthesizes meaningful neural activation patterns from open-ended descriptions

Demonstrates utility in literature search and hypothesis generation

Available as an accessible web-based tool

Abstract

Neuroimaging studies are often limited by the number of subjects and cognitive processes that can be feasibly interrogated. However, a rapidly growing number of neuroscientific studies have collectively accumulated an extensive wealth of results. Digesting this growing literature and obtaining novel insights remains to be a major challenge, since existing meta-analytic tools are constrained to keyword queries. In this paper, we present Text2Brain, an easy to use tool for synthesizing brain activation maps from open-ended text queries. Text2Brain was built on a transformer-based neural network language model and a coordinate-based meta-analysis of neuroimaging studies. Text2Brain combines a transformer-based text encoder and a 3D image generator, and was trained on variable-length text snippets and their corresponding activation maps sampled from 13,000 published studies. In our experiments, we demonstrate that Text2Brain can synthesize meaningful neural activation patterns from various free-form textual descriptions. Text2Brain is available at https://braininterpreter.com as a web-based tool for efficiently searching through the vast neuroimaging literature and generating new hypotheses.