Faithful Summarization of Consumer Health Queries: A Cross-Lingual Framework with LLMs

TL;DR

This paper introduces a cross-lingual framework combining extractive methods and LLMs to improve faithfulness in consumer health query summaries, reducing risks of misinformation in healthcare communication.

Contribution

It presents a novel framework that integrates TextRank, medical NER, and fine-tuned LLMs for faithful, multilingual health query summarization, outperforming previous approaches.

Findings

Achieved significant improvements in quality and faithfulness metrics.

Over 80% of summaries preserved critical medical information.

Demonstrated effectiveness across English and Bangla datasets.

Abstract

Summarizing consumer health questions (CHQs) can ease communication in healthcare, but unfaithful summaries that misrepresent medical details pose serious risks. We propose a framework that combines TextRank-based sentence extraction and medical named entity recognition with large language models (LLMs) to enhance faithfulness in medical text summarization. In our experiments, we fine-tuned the LLaMA-2-7B model on the MeQSum (English) and BanglaCHQ-Summ (Bangla) datasets, achieving consistent improvements across quality (ROUGE, BERTScore, readability) and faithfulness (SummaC, AlignScore) metrics, and outperforming zero-shot baselines and prior systems. Human evaluation further shows that over 80\% of generated summaries preserve critical medical information. These results highlight faithfulness as an essential dimension for reliable medical summarization and demonstrate the potential of our approach for safer deployment of LLMs in healthcare contexts.