mViSE: A Visual Search Engine for Analyzing Multiplex IHC Brain Tissue Images

TL;DR

mViSE is an open-source visual search engine that enables efficient, query-driven analysis of complex multiplex brain tissue images without programming, aiding tissue exploration and region delineation.

Contribution

It introduces a novel, programming-free approach using self-supervised learning to organize and retrieve multiplex brain tissue features through visual queries.

Findings

Successfully retrieves single cells and tissue patches

Delineates cortical layers and brain regions

Validated on diverse multiplex imaging data

Abstract

Whole-slide multiplex imaging of brain tissue generates massive information-dense images that are challenging to analyze and require custom software. We present an alternative query-driven programming-free strategy using a multiplex visual search engine (mViSE) that learns the multifaceted brain tissue chemoarchitecture, cytoarchitecture, and myeloarchitecture. Our divide-and-conquer strategy organizes the data into panels of related molecular markers and uses self-supervised learning to train a multiplex encoder for each panel with explicit visual confirmation of successful learning. Multiple panels can be combined to process visual queries for retrieving similar communities of individual cells or multicellular niches using information-theoretic methods. The retrievals can be used for diverse purposes including tissue exploration, delineating brain regions and cortical cell layers, profiling and comparing brain regions without computer programming. We validated mViSE's ability to retrieve single cells, proximal cell pairs, tissue patches, delineate cortical layers, brain regions and sub-regions. mViSE is provided as an open-source QuPath plug-in.