LYSTO: The Lymphocyte Assessment Hackathon and Benchmark Dataset

TL;DR

LYSTO is a hackathon and benchmark dataset for automated lymphocyte assessment in cancer histopathology images, demonstrating that some methods can reach pathologist-level accuracy and providing a resource for ongoing research and education.

Contribution

This paper introduces LYSTO, a novel hackathon and benchmark dataset for lymphocyte assessment, with methods achieving pathologist-level performance and supporting future research.

Findings

Some methods achieved pathologist-level accuracy.

LYSTO serves as a lightweight, plug-and-play benchmark dataset.

The dataset supports ongoing research and education in digital pathology.

Abstract

We introduce LYSTO, the Lymphocyte Assessment Hackathon, which was held in conjunction with the MICCAI 2019 Conference in Shenzen (China). The competition required participants to automatically assess the number of lymphocytes, in particular T-cells, in histopathological images of colon, breast, and prostate cancer stained with CD3 and CD8 immunohistochemistry. Differently from other challenges setup in medical image analysis, LYSTO participants were solely given a few hours to address this problem. In this paper, we describe the goal and the multi-phase organization of the hackathon; we describe the proposed methods and the on-site results. Additionally, we present post-competition results where we show how the presented methods perform on an independent set of lung cancer slides, which was not part of the initial competition, as well as a comparison on lymphocyte assessment between presented methods and a panel of pathologists. We show that some of the participants were capable to achieve pathologist-level performance at lymphocyte assessment. After the hackathon, LYSTO was left as a lightweight plug-and-play benchmark dataset on grand-challenge website, together with an automatic evaluation platform. LYSTO has supported a number of research in lymphocyte assessment in oncology. LYSTO will be a long-lasting educational challenge for deep learning and digital pathology, it is available at https://lysto.grand-challenge.org/.