Tackling Hallucinations in Neural Chart Summarization

TL;DR

This paper addresses hallucinations in neural chart summarization by analyzing dataset issues and proposing an NLI-based preprocessing method, which, along with input modifications, reduces hallucinations and improves summarization quality.

Contribution

It introduces an NLI-based data preprocessing approach to mitigate hallucinations in neural chart summarization, a novel application in this domain.

Findings

NLI preprocessing significantly reduces hallucinations.

Shortening dependencies improves summarization.

Adding chart metadata enhances performance.

Abstract

Hallucinations in text generation occur when the system produces text that is not grounded in the input. In this work, we tackle the problem of hallucinations in neural chart summarization. Our analysis shows that the target side of chart summarization training datasets often contains additional information, leading to hallucinations. We propose a natural language inference (NLI) based method to preprocess the training data and show through human evaluation that our method significantly reduces hallucinations. We also found that shortening long-distance dependencies in the input sequence and adding chart-related information like title and legends improves the overall performance.