Conditional Random Fields for Interactive Refinement of Histopathological Predictions

TL;DR

This paper introduces HistoCRF, a CRF-based framework that refines Vision-Language Model predictions for histopathological image analysis, significantly improving accuracy with minimal annotations and human-in-the-loop corrections.

Contribution

It presents a novel CRF-based method tailored for histopathology that enhances zero-shot predictions without additional training, incorporating expert annotations and iterative human feedback.

Findings

Average accuracy gain of 16.0% without annotations

27.5% accuracy improvement with 100 annotations

Further 32.6% gain with human-in-the-loop corrections

Abstract

Assisting pathologists in the analysis of histopathological images has high clinical value, as it supports cancer detection and staging. In this context, histology foundation models have recently emerged. Among them, Vision-Language Models (VLMs) provide strong yet imperfect zero-shot predictions. We propose to refine these predictions by adapting Conditional Random Fields (CRFs) to histopathological applications, requiring no additional model training. We present HistoCRF, a CRF-based framework, with a novel definition of the pairwise potential that promotes label diversity and leverages expert annotations. We consider three experiments: without annotations, with expert annotations, and with iterative human-in-the-loop annotations that progressively correct misclassified patches. Experiments on five patch-level classification datasets covering different organs and diseases demonstrate average accuracy gains of 16.0% without annotations and 27.5% with only 100 annotations, compared to zero-shot predictions. Moreover, integrating a human in the loop reaches a further gain of 32.6% with the same number of annotations. The code will be made available on https://github.com/tgodelaine/HistoCRF.