Semantic Mosaicing of Histo-Pathology Image Fragments using Visual Foundation Models

TL;DR

This paper presents SemanticStitcher, a novel method leveraging visual foundation models to improve the robustness and accuracy of stitching histopathology image fragments into whole mount slides, overcoming challenges like tissue loss and distortion.

Contribution

Introduction of SemanticStitcher, a new semantic-based stitching method using latent features from foundation models for better tissue fragment alignment.

Findings

Outperforms existing boundary shape matching methods in accuracy.

Demonstrates robustness across three diverse histopathology datasets.

Achieves consistent improvement in mosaic quality and boundary matching.

Abstract

In histopathology, tissue samples are often larger than a standard microscope slide, making stitching of multiple fragments necessary to process entire structures such as tumors. Automated stitching is a prerequisite for scaling analysis, but is challenging due to possible tissue loss during preparation, inhomogeneous morphological distortion, staining inconsistencies, missing regions due to misalignment on the slide, or frayed tissue edges. This limits state-of-the-art stitching methods using boundary shape matching algorithms to reconstruct artificial whole mount slides (WMS). Here, we introduce SemanticStitcher using latent feature representations derived from a visual histopathology foundation model to identify neighboring areas in different fragments. Robust pose estimation based on a large number of semantic matching candidates derives a mosaic of multiple fragments to form the WMS. Experiments on three different histopathology datasets demonstrate that SemanticStitcher yields robust WMS mosaicing and consistently outperforms the state of the art in correct boundary matches.