AI Assisted Annotator using Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning approach to automate medical data annotation, mimicking expert decision-making to improve accuracy and reduce costs in healthcare data labeling.

Contribution

It presents the first application of reinforcement learning for automating medical event annotation, demonstrating promising results with DQN and A2C agents.

Findings

A2C outperforms DQN in learning sparse medical events

RL agents successfully mimic expert annotation behavior

Initial results show potential for practical healthcare applications

Abstract

Healthcare data suffers from both noise and lack of ground truth. The cost of data increases as it is cleaned and annotated in healthcare. Unlike other data sets, medical data annotation, which is critical to accurate ground truth, requires medical domain expertise for a better patient outcome. In this work, we report on the use of reinforcement learning to mimic the decision making process of annotators for medical events, to automate annotation and labelling. The reinforcement agent learns to annotate alarm data based on annotations done by an expert. Our method shows promising results on medical alarm data sets. We trained DQN and A2C agents using the data from monitoring devices annotated by an expert. Initial results from these RL agents learning the expert annotation behavior are promising. The A2C agent performs better in terms of learning the sparse events in a given state, thereby choosing more right actions compared to DQN agent. To the best of our knowledge, this is the first reinforcement learning application for the automation of medical events annotation, which has far-reaching practical use.