Clinical Reading Comprehension with Encoder-Decoder Models Enhanced by Direct Preference Optimization

TL;DR

This paper enhances clinical reading comprehension by combining encoder-decoder models with direct preference optimization, achieving significant improvements in radiology question answering and pioneering the use of DPO without human-labeled data.

Contribution

It introduces a novel application of DPO to encoder-decoder models for clinical reading comprehension, improving state-of-the-art performance and demonstrating DPO's effectiveness without human preference data.

Findings

Achieved 12-15 F1 point improvements on RadQA.

First to apply DPO to reading comprehension tasks.

Demonstrated DPO's effectiveness without human-labeled preferences.

Abstract

Extractive question answering over clinical text is a crucial need to help deal with the deluge of clinical text generated in hospitals. While encoder models (e.g., BERT) have been popular for this reading comprehension task, recently encoder-decoder models (e.g., T5) are on the rise. There is also the emergence of preference optimization techniques to align decoder-only LLMs with human preferences. In this paper, we combine encoder-decoder models with the direct preference optimization (DPO) method to improve over prior state of the art for the RadQA radiology question answering task by 12-15 F1 points. To the best of our knowledge, this effort is the first to show that DPO method also works for reading comprehension via novel heuristics to generate preference data without human inputs.