Procedural Text Understanding via Scene-Wise Evolution

TL;DR

This paper introduces a scene-wise paradigm and Scene Graph Reasoner (SGR) for procedural text understanding, enabling joint tracking of entity states and interactions, leading to improved accuracy and faster reasoning.

Contribution

It proposes a novel scene-wise approach with dynamic scene graphs, advancing beyond entity-wise methods for more integrated procedural text reasoning.

Findings

SGR achieves state-of-the-art performance.

SGR significantly accelerates reasoning speed.

Joint modeling of entities and states improves understanding.

Abstract

Procedural text understanding requires machines to reason about entity states within the dynamical narratives. Current procedural text understanding approaches are commonly \textbf{entity-wise}, which separately track each entity and independently predict different states of each entity. Such an entity-wise paradigm does not consider the interaction between entities and their states. In this paper, we propose a new \textbf{scene-wise} paradigm for procedural text understanding, which jointly tracks states of all entities in a scene-by-scene manner. Based on this paradigm, we propose \textbf{S}cene \textbf{G}raph \textbf{R}easoner (\textbf{SGR}), which introduces a series of dynamically evolving scene graphs to jointly formulate the evolution of entities, states and their associations throughout the narrative. In this way, the deep interactions between all entities and states can be jointly captured and simultaneously derived from scene graphs. Experiments show that SGR not only achieves the new state-of-the-art performance but also significantly accelerates the speed of reasoning.