Modeling Attention Flow on Graphs

TL;DR

This paper introduces an attention flow mechanism on graphs that improves trajectory reasoning by modeling information flow, leading to better prediction accuracy and interpretability in complex systems.

Contribution

It proposes a novel attention flow mechanism integrated with graph networks for explicit reasoning on graph-structured data.

Findings

Attention flow enhances prediction accuracy.

The method provides clearer interpretability.

Effective in trajectory reasoning tasks.

Abstract

Real-world scenarios demand reasoning about process, more than final outcome prediction, to discover latent causal chains and better understand complex systems. It requires the learning algorithms to offer both accurate predictions and clear interpretations. We design a set of trajectory reasoning tasks on graphs with only the source and the destination observed. We present the attention flow mechanism to explicitly model the reasoning process, leveraging the relational inductive biases by basing our models on graph networks. We study the way attention flow can effectively act on the underlying information flow implemented by message passing. Experiments demonstrate that the attention flow driven by and interacting with graph networks can provide higher accuracy in prediction and better interpretation for trajectory reasoning.