Context-Dependent Semantic Parsing over Temporally Structured Data

TL;DR

This paper introduces a context-aware semantic parsing framework for querying temporally structured data, combining natural language and GUI actions, with an LSTM-based model that outperforms standard baselines.

Contribution

It presents a novel semantic parsing setting integrating GUI actions and natural language, and an LSTM-based architecture with copying and attention mechanisms for context modeling.

Findings

Achieved 88.7% sequence accuracy on artificial data.

Achieved 74.8% sequence accuracy on real data.

Supervised training outperforms standard sequence generation baselines.

Abstract

We describe a new semantic parsing setting that allows users to query the system using both natural language questions and actions within a graphical user interface. Multiple time series belonging to an entity of interest are stored in a database and the user interacts with the system to obtain a better understanding of the entity's state and behavior, entailing sequences of actions and questions whose answers may depend on previous factual or navigational interactions. We design an LSTM-based encoder-decoder architecture that models context dependency through copying mechanisms and multiple levels of attention over inputs and previous outputs. When trained to predict tokens using supervised learning, the proposed architecture substantially outperforms standard sequence generation baselines. Training the architecture using policy gradient leads to further improvements in performance, reaching a sequence-level accuracy of 88.7% on artificial data and 74.8% on real data.