Towards Understanding of Medical Randomized Controlled Trials by Conclusion Generation

TL;DR

This paper explores automatic conclusion generation for medical RCTs using sequence-to-sequence models, demonstrating that fine-tuned GPT-2 improves conclusion quality over baseline models, aiding clinical decision-making.

Contribution

It introduces a novel task of RCT conclusion generation and demonstrates the effectiveness of fine-tuned GPT-2 models in this domain.

Findings

GPT-2 fine-tuned models outperform baseline models in conclusion quality

Automatic and human evaluations confirm improved correctness of generated conclusions

Identifies limitations and future directions for medical RCT conclusion generation

Abstract

Randomized controlled trials (RCTs) represent the paramount evidence of clinical medicine. Using machines to interpret the massive amount of RCTs has the potential of aiding clinical decision-making. We propose a RCT conclusion generation task from the PubMed 200k RCT sentence classification dataset to examine the effectiveness of sequence-to-sequence models on understanding RCTs. We first build a pointer-generator baseline model for conclusion generation. Then we fine-tune the state-of-the-art GPT-2 language model, which is pre-trained with general domain data, for this new medical domain task. Both automatic and human evaluation show that our GPT-2 fine-tuned models achieve improved quality and correctness in the generated conclusions compared to the baseline pointer-generator model. Further inspection points out the limitations of this current approach and future directions to explore.