Learning from the Right Patches: A Two-Stage Wavelet-Driven Masked Autoencoder for Histopathology Representation Learning

TL;DR

This paper introduces WISE-MAE, a wavelet-informed patch selection method for masked autoencoders that enhances histopathology image representation learning by focusing on biologically relevant tissue regions.

Contribution

The paper proposes a novel two-stage wavelet-driven patch selection strategy for MAEs, improving relevance and efficiency in histopathology representation learning.

Findings

Achieves competitive classification performance across multiple cancer datasets.

Improves the quality of learned representations by focusing on structurally rich regions.

Maintains efficiency under weak supervision.

Abstract

Whole-slide images are central to digital pathology, yet their extreme size and scarce annotations make self-supervised learning essential. Masked Autoencoders (MAEs) with Vision Transformer backbones have recently shown strong potential for histopathology representation learning. However, conventional random patch sampling during MAE pretraining often includes irrelevant or noisy regions, limiting the model's ability to capture meaningful tissue patterns. In this paper, we present a lightweight and domain-adapted framework that brings structure and biological relevance into MAE-based learning through a wavelet-informed patch selection strategy. WISE-MAE applies a two-step coarse-to-fine process: wavelet-based screening at low magnification to locate structurally rich regions, followed by high-resolution extraction for detailed modeling. This approach mirrors the diagnostic workflow of pathologists and improves the quality of learned representations. Evaluations across multiple cancer datasets, including lung, renal, and colorectal tissues, show that WISE-MAE achieves competitive representation quality and downstream classification performance while maintaining efficiency under weak supervision.