Timo: Towards Better Temporal Reasoning for Language Models

TL;DR

Timo is a new large language model designed specifically to improve temporal reasoning across a wide range of tasks, using a novel self-critic optimization method to achieve state-of-the-art results.

Contribution

The paper introduces Timo, a model that enhances temporal reasoning in LLMs through a self-critic optimization, demonstrating improved performance across diverse tasks.

Findings

Timo outperforms comparable models by 7.6-10.0 in accuracy.

Self-critic temporal optimization improves reasoning capabilities.

Effective across diverse temporal reasoning tasks.

Abstract

Reasoning about time is essential for Large Language Models (LLMs) to understand the world. Previous works focus on solving specific tasks, primarily on time-sensitive question answering. While these methods have proven effective, they cannot generalize to a wider spectrum of temporal reasoning tasks. Therefore, we propose a crucial question: Can we build a universal framework to handle a variety of temporal reasoning tasks? To that end, we systematically study 38 temporal reasoning tasks. Based on the observation that 19 tasks are directly related to mathematics, we first leverage the available mathematical dataset to set a solid foundation for temporal reasoning. However, the in-depth study indicates that focusing solely on mathematical enhancement falls short of addressing pure temporal reasoning tasks. To mitigate this limitation, we propose a simple but effective self-critic temporal optimization method to enhance the model's temporal reasoning capabilities without sacrificing general task abilities. Finally, we develop Timo, a model designed to excel in temporal reasoning at the 7B and 13B scales. Notably, Timo outperforms the counterpart LLMs by 10.0 and 7.6 in average accuracy scores and achieves the new state-of-the-art (SOTA) performance of comparable size. Extensive experiments further validate our framework's effectiveness and its generalization across diverse temporal tasks. The code is available at https://github.com/zhaochen0110/Timo.