Quality monitoring of federated Covid-19 lesion segmentation

TL;DR

This paper introduces lightweight, locally computable metrics for federated monitoring of Covid-19 lesion segmentation quality in chest CTs, enabling effective performance tracking without extensive expert review.

Contribution

It proposes a set of lightweight metrics for local computation and central aggregation to monitor federated model performance in Covid-19 lesion segmentation.

Findings

Linear model detects over 70% of low-quality segmentations

Metrics reliably signal performance decline

Enables continuous, privacy-preserving quality monitoring

Abstract

Federated Learning is the most promising way to train robust Deep Learning models for the segmentation of Covid-19-related findings in chest CTs. By learning in a decentralized fashion, heterogeneous data can be leveraged from a variety of sources and acquisition protocols whilst ensuring patient privacy. It is, however, crucial to continuously monitor the performance of the model. Yet when it comes to the segmentation of diffuse lung lesions, a quick visual inspection is not enough to assess the quality, and thorough monitoring of all network outputs by expert radiologists is not feasible. In this work, we present an array of lightweight metrics that can be calculated locally in each hospital and then aggregated for central monitoring of a federated system. Our linear model detects over 70% of low-quality segmentations on an out-of-distribution dataset and thus reliably signals a decline in model performance.