Hierarchical Evidence Set Modeling for Automated Fact Extraction and Verification

TL;DR

This paper introduces Hierarchical Evidence Set Modeling (HESM), a novel framework that improves automated fact verification by effectively combining related evidence sentences at multiple hierarchy levels, outperforming existing methods.

Contribution

The paper proposes a hierarchical approach to evidence set modeling that enhances the accuracy of fact extraction and claim verification over prior concatenation or separate processing methods.

Findings

HESM outperforms 7 state-of-the-art models in fact verification tasks.

Hierarchical encoding improves the integration of related evidence sentences.

The approach reduces noise and redundancy in evidence selection.

Abstract

Automated fact extraction and verification is a challenging task that involves finding relevant evidence sentences from a reliable corpus to verify the truthfulness of a claim. Existing models either (i) concatenate all the evidence sentences, leading to the inclusion of redundant and noisy information; or (ii) process each claim-evidence sentence pair separately and aggregate all of them later, missing the early combination of related sentences for more accurate claim verification. Unlike the prior works, in this paper, we propose Hierarchical Evidence Set Modeling (HESM), a framework to extract evidence sets (each of which may contain multiple evidence sentences), and verify a claim to be supported, refuted or not enough info, by encoding and attending the claim and evidence sets at different levels of hierarchy. Our experimental results show that HESM outperforms 7 state-of-the-art methods for fact extraction and claim verification. Our source code is available at https://github.com/ShyamSubramanian/HESM.