Tackling Hallucinations in Neural Chart Summarization
Saad Obaid ul Islam, Iza \v{S}krjanec, Ond\v{r}ej Du\v{s}ek, Vera, Demberg

TL;DR
This paper addresses hallucinations in neural chart summarization by analyzing dataset issues and proposing an NLI-based preprocessing method, which, along with input modifications, reduces hallucinations and improves summarization quality.
Contribution
It introduces an NLI-based data preprocessing approach to mitigate hallucinations in neural chart summarization, a novel application in this domain.
Findings
NLI preprocessing significantly reduces hallucinations.
Shortening dependencies improves summarization.
Adding chart metadata enhances performance.
Abstract
Hallucinations in text generation occur when the system produces text that is not grounded in the input. In this work, we tackle the problem of hallucinations in neural chart summarization. Our analysis shows that the target side of chart summarization training datasets often contains additional information, leading to hallucinations. We propose a natural language inference (NLI) based method to preprocess the training data and show through human evaluation that our method significantly reduces hallucinations. We also found that shortening long-distance dependencies in the input sequence and adding chart-related information like title and legends improves the overall performance.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Advanced Text Analysis Techniques · Natural Language Processing Techniques
