Deep Learning for BioImaging: What Are We Learning?

TL;DR

This paper systematically evaluates representation learning methods in microscopy imaging, revealing that current models often do not learn meaningful biological features and highlighting the need for better benchmarks.

Contribution

It introduces simple baseline models and benchmarks, demonstrating that state-of-the-art methods do not outperform them significantly in microscopy image analysis.

Findings

State-of-the-art methods perform similarly to simple baselines.

Existing models fail to learn high-level biological features.

Current benchmarks are insufficient to evaluate representation quality.

Abstract

Representation learning has driven major advances in natural image analysis by enabling models to acquire high-level semantic features. In microscopy imaging, however, it remains unclear what current representation learning methods actually learn. In this work, we conduct a systematic study of representation learning for the two most widely used and broadly available microscopy data types, representing critical scales in biology: cell culture and tissue imaging. To this end, we introduce a set of simple yet revealing baselines on curated benchmarks, including untrained models and simple structural representations of cellular tissue. Our results show that, surprisingly, state-of-the-art methods perform comparably to these baselines. We further show that, in contrast to natural images, existing models fail to consistently acquire high-level, biologically meaningful features. Moreover, we demonstrate that commonly used benchmark metrics are insufficient to assess representation quality and often mask this limitation. In addition, we investigate how detailed comparisons with these benchmarks provide ways to interpret the strengths and weaknesses of models for further improvements. Together, our results suggest that progress in microscopy image representation learning requires not only stronger models, but also more diagnostic benchmarks that measure what is actually learned.