Multi-environment lifelong deep reinforcement learning for medical imaging

TL;DR

This paper introduces SERIL, a lifelong deep reinforcement learning framework that effectively learns multiple medical imaging tasks across changing environments without forgetting previous tasks, demonstrated on brain MRI landmark localization.

Contribution

The paper presents SERIL, a novel lifelong DRL method using selective experience replay to handle evolving medical imaging environments and tasks.

Findings

SERIL outperforms baseline methods in landmark localization accuracy.

It maintains high performance across 120 diverse imaging environments.

The approach prevents catastrophic forgetting in dynamic medical imaging tasks.

Abstract

Deep reinforcement learning(DRL) is increasingly being explored in medical imaging. However, the environments for medical imaging tasks are constantly evolving in terms of imaging orientations, imaging sequences, and pathologies. To that end, we developed a Lifelong DRL framework, SERIL to continually learn new tasks in changing imaging environments without catastrophic forgetting. SERIL was developed using selective experience replay based lifelong learning technique for the localization of five anatomical landmarks in brain MRI on a sequence of twenty-four different imaging environments. The performance of SERIL, when compared to two baseline setups: MERT(multi-environment-best-case) and SERT(single-environment-worst-case) demonstrated excellent performance with an average distance of $9.90\pm7.35$ pixels from the desired landmark across all 120 tasks, compared to $10.29\pm9.07$ for MERT and $36.37\pm22.41$ for SERT($p<0.05$), demonstrating the excellent potential for continuously learning multiple tasks across dynamically changing imaging environments.