Semantic annotation for computational pathology: Multidisciplinary experience and best practice recommendations

TL;DR

This paper shares best practices and guidelines for annotating pathology images to improve the accuracy and interpretability of AI algorithms in computational pathology, based on a large-scale multidisciplinary case study.

Contribution

It provides the first comprehensive set of annotation guidelines for computational pathology, developed from real-world experience with multidisciplinary teams.

Findings

Established annotation best practices for CPath projects

Developed a detailed annotation data dictionary and constructs

Highlighted the impact of proper annotations on algorithm performance

Abstract

Recent advances in whole slide imaging (WSI) technology have led to the development of a myriad of computer vision and artificial intelligence (AI) based diagnostic, prognostic, and predictive algorithms. Computational Pathology (CPath) offers an integrated solution to utilize information embedded in pathology WSIs beyond what we obtain through visual assessment. For automated analysis of WSIs and validation of machine learning (ML) models, annotations at the slide, tissue and cellular levels are required. The annotation of important visual constructs in pathology images is an important component of CPath projects. Improper annotations can result in algorithms which are hard to interpret and can potentially produce inaccurate and inconsistent results. Despite the crucial role of annotations in CPath projects, there are no well-defined guidelines or best practices on how annotations should be carried out. In this paper, we address this shortcoming by presenting the experience and best practices acquired during the execution of a large-scale annotation exercise involving a multidisciplinary team of pathologists, ML experts and researchers as part of the Pathology image data Lake for Analytics, Knowledge and Education (PathLAKE) consortium. We present a real-world case study along with examples of different types of annotations, diagnostic algorithm, annotation data dictionary and annotation constructs. The analyses reported in this work highlight best practice recommendations that can be used as annotation guidelines over the lifecycle of a CPath project.