Offline versus Online Triplet Mining based on Extreme Distances of Histopathology Patches

TL;DR

This paper compares offline and online triplet mining methods for histopathology image embedding, analyzing the impact of extreme distances and neighbor patches, and finds comparable performance with promising results in online approaches.

Contribution

It provides a comprehensive analysis of offline versus online triplet mining using extreme distances in histopathology data, highlighting their relative effectiveness and theoretical connections.

Findings

Offline and online mining have similar performance with ResNet-18.

Extreme distance-based approaches show promise, especially online.

Offline mining can be viewed as a generalization of online mining with large batch sizes.

Abstract

We analyze the effect of offline and online triplet mining for colorectal cancer (CRC) histopathology dataset containing 100,000 patches. We consider the extreme, i.e., farthest and nearest patches to a given anchor, both in online and offline mining. While many works focus solely on selecting the triplets online (batch-wise), we also study the effect of extreme distances and neighbor patches before training in an offline fashion. We analyze extreme cases' impacts in terms of embedding distance for offline versus online mining, including easy positive, batch semi-hard, batch hard triplet mining, neighborhood component analysis loss, its proxy version, and distance weighted sampling. We also investigate online approaches based on extreme distance and comprehensively compare offline, and online mining performance based on the data patterns and explain offline mining as a tractable generalization of the online mining with large mini-batch size. As well, we discuss the relations of different colorectal tissue types in terms of extreme distances. We found that offline and online mining approaches have comparable performances for a specific architecture, such as ResNet-18 in this study. Moreover, we found the assorted case, including different extreme distances, is promising, especially in the online approach.