Counterfactual-Consistency Prompting for Relative Temporal Understanding in Large Language Models

TL;DR

This paper introduces a counterfactual prompting method to improve large language models' temporal reasoning and consistency, significantly enhancing their ability to correctly understand and order events in time.

Contribution

It proposes a novel counterfactual prompting approach that enforces temporal consistency in LLMs, addressing a key limitation in their reasoning capabilities.

Findings

Improved event ordering accuracy in LLMs

Enhanced temporal commonsense understanding

Significant reduction in temporal inconsistencies

Abstract

Despite the advanced capabilities of large language models (LLMs), their temporal reasoning ability remains underdeveloped. Prior works have highlighted this limitation, particularly in maintaining temporal consistency when understanding events. For example, models often confuse mutually exclusive temporal relations like ``before'' and ``after'' between events and make inconsistent predictions. In this work, we tackle the issue of temporal inconsistency in LLMs by proposing a novel counterfactual prompting approach. Our method generates counterfactual questions and enforces collective constraints, enhancing the model's consistency. We evaluate our method on multiple datasets, demonstrating significant improvements in event ordering for explicit and implicit events and temporal commonsense understanding by effectively addressing temporal inconsistencies.