Analysis and Validation of Image Search Engines in Histopathology

TL;DR

This study extensively analyzes and validates various image search engines for histopathology, highlighting their strengths and weaknesses in accuracy, efficiency, and storage, to improve patient matching and diagnosis.

Contribution

The paper provides a comprehensive evaluation of four histopathology image search methods, comparing their algorithms, performance, and potential variants across large datasets.

Findings

BoVW is fast but less accurate

Yottixel balances speed and accuracy

SISH and RetCCL show limitations in efficiency and accuracy

Abstract

Searching for similar images in archives of histology and histopathology images is a crucial task that may aid in patient matching for various purposes, ranging from triaging and diagnosis to prognosis and prediction. Whole slide images (WSIs) are highly detailed digital representations of tissue specimens mounted on glass slides. Matching WSI to WSI can serve as the critical method for patient matching. In this paper, we report extensive analysis and validation of four search methods bag of visual words (BoVW), Yottixel, SISH, RetCCL, and some of their potential variants. We analyze their algorithms and structures and assess their performance. For this evaluation, we utilized four internal datasets ($1269$ patients) and three public datasets ($1207$ patients), totaling more than $200,000$ patches from $38$ different classes/subtypes across five primary sites. Certain search engines, for example, BoVW, exhibit notable efficiency and speed but suffer from low accuracy. Conversely, search engines like Yottixel demonstrate efficiency and speed, providing moderately accurate results. Recent proposals, including SISH, display inefficiency and yield inconsistent outcomes, while alternatives like RetCCL prove inadequate in both accuracy and efficiency. Further research is imperative to address the dual aspects of accuracy and minimal storage requirements in histopathological image search.