Review learning: Real world validation of privacy preserving continual learning across medical institutions

TL;DR

This paper introduces 'review learning' (RevL), a novel continual learning algorithm that effectively mitigates catastrophic forgetting in privacy-preserving medical diagnosis models trained on electronic health records across multiple institutions.

Contribution

RevL is a low-cost continual learning method that reviews previous knowledge by generating data samples, validated through multiple simulated and real-world experiments in medical settings.

Findings

RevL outperforms transfer learning in retaining knowledge.

In real-world data, RevL achieved a higher AUC (0.710) than transfer learning (0.655).

RevL demonstrates practical applicability for privacy-preserving continual learning in healthcare.

Abstract

When a deep learning model is trained sequentially on different datasets, it often forgets the knowledge learned from previous data, a problem known as catastrophic forgetting. This damages the model's performance on diverse datasets, which is critical in privacy-preserving deep learning (PPDL) applications based on transfer learning (TL). To overcome this, we introduce "review learning" (RevL), a low cost continual learning algorithm for diagnosis prediction using electronic health records (EHR) within a PPDL framework. RevL generates data samples from the model which are used to review knowledge from previous datasets. Six simulated institutional experiments and one real-world experiment involving three medical institutions were conducted to validate RevL, using three binary classification EHR data. In the real-world experiment with data from 106,508 patients, the mean global area under the receiver operating curve was 0.710 for RevL and 0.655 for TL. These results demonstrate RevL's ability to retain previously learned knowledge and its effectiveness in real-world PPDL scenarios. Our work establishes a realistic pipeline for PPDL research based on model transfers across institutions and highlights the practicality of continual learning in real-world medical settings using private EHR data.