Mitigating Hallucinations in Zero-Shot Scientific Summarisation: A Pilot Study
Imane Jaaouine, Ross D. King

TL;DR
This study explores prompt engineering techniques to reduce hallucinations in zero-shot scientific summarisation by LLMs, demonstrating that context repetition and random addition improve lexical alignment with scientific abstracts.
Contribution
It provides empirical evidence that specific prompt engineering methods can mitigate hallucinations in zero-shot LLM summarisation of scientific texts.
Findings
Context repetition improves lexical alignment.
Random addition enhances summary relevance.
Prompt engineering can reduce hallucinations.
Abstract
Large language models (LLMs) produce context inconsistency hallucinations, which are LLM generated outputs that are misaligned with the user prompt. This research project investigates whether prompt engineering (PE) methods can mitigate context inconsistency hallucinations in zero-shot LLM summarisation of scientific texts, where zero-shot indicates that the LLM relies purely on its pre-training data. Across eight yeast biotechnology research paper abstracts, six instruction-tuned LLMs were prompted with seven methods: a baseline prompt, two levels of increasing instruction complexity (PE-1 and PE-2), two levels of context repetition (CR-K1 and CR-K2), and two levels of random addition (RA-K1 and RA-K2). Context repetition involved the identification and repetition of K key sentences from the abstract, whereas random addition involved the repetition of K randomly selected sentences from…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBiomedical Text Mining and Ontologies · Topic Modeling · Text Readability and Simplification
