ViTally Consistent: Scaling Biological Representation Learning for Cell Microscopy

TL;DR

This paper introduces the largest foundation model for cell microscopy, a 1.9 billion-parameter ViT-G/8 MAE trained on over 8 billion images, significantly improving biological representation consistency and downstream analysis.

Contribution

It presents a new large-scale ViT model trained on curated microscopy data, with methods to optimize biological feature representations and insights into transformer block utility.

Findings

60% improvement in genetic perturbation separability

Best performance on biological relationship benchmarks

Intermediate transformer blocks yield more meaningful biological features

Abstract

Large-scale cell microscopy screens are used in drug discovery and molecular biology research to study the effects of millions of chemical and genetic perturbations on cells. To use these images in downstream analysis, we need models that can map each image into a feature space that represents diverse biological phenotypes consistently, in the sense that perturbations with similar biological effects have similar representations. In this work, we present the largest foundation model for cell microscopy data to date, a new 1.9 billion-parameter ViT-G/8 MAE trained on over 8 billion microscopy image crops. Compared to a previous published ViT-L/8 MAE, our new model achieves a 60% improvement in linear separability of genetic perturbations and obtains the best overall performance on whole-genome biological relationship recall and replicate consistency benchmarks. Beyond scaling, we developed two key methods that improve performance: (1) training on a curated and diverse dataset; and, (2) using biologically motivated linear probing tasks to search across each transformer block for the best candidate representation of whole-genome screens. We find that many self-supervised vision transformers, pretrained on either natural or microscopy images, yield significantly more biologically meaningful representations of microscopy images in their intermediate blocks than in their typically used final blocks. More broadly, our approach and results provide insights toward a general strategy for successfully building foundation models for large-scale biological data.