MONET -- Virtual Cell Painting of Brightfield Images and Time Lapses Using Reference Consistent Diffusion

TL;DR

This paper introduces MONET, a diffusion model that predicts cell paint images from brightfield images, enabling virtual cell painting and time-lapse video generation without chemical fixation, thus facilitating dynamic cell studies.

Contribution

The paper presents a novel diffusion-based model with a consistency architecture for virtual cell painting and time-lapse video synthesis from brightfield images.

Findings

Model quality improves with scale.

The architecture enables time-lapse video generation.

Partial transfer to new cell lines and protocols is possible.

Abstract

Cell painting is a popular technique for creating human-interpretable, high-contrast images of cell morphology. There are two major issues with cell paint: (1) it is labor-intensive and (2) it requires chemical fixation, making the study of cell dynamics impossible. We train a diffusion model (Morphological Observation Neural Enhancement Tool, or MONET) on a large dataset to predict cell paint channels from brightfield images. We show that model quality improves with scale. The model uses a consistency architecture to generate time-lapse videos, despite the impossibility of obtaining cell paint video training data. In addition, we show that this architecture enables a form of in-context learning, allowing the model to partially transfer to out-of-distribution cell lines and imaging protocols. Virtual cell painting is not intended to replace physical cell painting completely, but to act as a complementary tool enabling novel workflows in biological research.