TL;DR
FACTTRACK is a novel method for tracking atomic facts with time-aware validity intervals to detect and correct factual contradictions in story outlines, improving accuracy over existing models and maintaining dynamic world states.
Contribution
The paper introduces FACTTRACK, a new approach that incorporates time-aware validity intervals for facts, enhancing contradiction detection in narrative structures.
Findings
FACTTRACK outperforms baseline models in contradiction detection.
FACTTRACK achieves performance comparable to GPT-4 with LLaMA2-7B-Chat.
Using GPT-4, FACTTRACK significantly surpasses baseline performance.
Abstract
While accurately detecting and correcting factual contradictions in language model outputs has become increasingly important as their capabilities improve, doing so is highly challenging. We propose a novel method, FACTTRACK, for tracking atomic facts and addressing factual contradictions. Crucially, FACTTRACK also maintains time-aware validity intervals for each fact, allowing for change over time. At a high level, FACTTRACK consists of a four-step pipeline to update a world state data structure for each new event: (1) decompose the event into directional atomic facts; (2) determine the validity interval of each atomic fact using the world state; (3) detect contradictions with existing facts in the world state; and finally (4) add new facts to the world state and update existing atomic facts. When we apply FACTTRACK to contradiction detection on structured story outlines, we find that…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
