The Lay Person's Guide to Biomedicine: Orchestrating Large Language Models

TL;DR

This paper explores using large language models to generate and evaluate lay summaries of biomedical articles, introducing a new framework and metrics that improve accessibility and assessment aligned with human preferences.

Contribution

It proposes an Explain-then-Summarise framework leveraging LLMs for background knowledge and introduces novel LLM-based evaluation metrics for lay summaries.

Findings

LLM-generated background knowledge enhances supervised lay summarisation.

The zero-shot evaluation metric aligns well with human preferences.

Human assessment confirms the quality of LLM-generated summaries.

Abstract

Automated lay summarisation (LS) aims to simplify complex technical documents into a more accessible format to non-experts. Existing approaches using pre-trained language models, possibly augmented with external background knowledge, tend to struggle with effective simplification and explanation. Moreover, automated methods that can effectively assess the `layness' of generated summaries are lacking. Recently, large language models (LLMs) have demonstrated a remarkable capacity for text simplification, background information generation, and text evaluation. This has motivated our systematic exploration into using LLMs to generate and evaluate lay summaries of biomedical articles. We propose a novel \textit{Explain-then-Summarise} LS framework, which leverages LLMs to generate high-quality background knowledge to improve supervised LS. We also evaluate the performance of LLMs for zero-shot LS and propose two novel LLM-based LS evaluation metrics, which assess layness from multiple perspectives. Finally, we conduct a human assessment of generated lay summaries. Our experiments reveal that LLM-generated background information can support improved supervised LS. Furthermore, our novel zero-shot LS evaluation metric demonstrates a high degree of alignment with human preferences. We conclude that LLMs have an important part to play in improving both the performance and evaluation of LS methods.