Intrinsically motivated graph exploration using network theories of human curiosity

TL;DR

This paper introduces a novel intrinsic motivation approach for graph exploration based on human curiosity theories, improving exploration efficiency and prediction of human choices in graph environments.

Contribution

It proposes a new curiosity-driven exploration method for graph-structured data using theories of human curiosity, with neural network-based rewards and demonstrated generalization.

Findings

Agents generalize to longer walks and larger graphs.

Method computes topological features more efficiently than greedy evaluation.

Curiosity-based recommendations outperform PageRank in predicting human choices.

Abstract

Intrinsically motivated exploration has proven useful for reinforcement learning, even without additional extrinsic rewards. When the environment is naturally represented as a graph, how to guide exploration best remains an open question. In this work, we propose a novel approach for exploring graph-structured data motivated by two theories of human curiosity: the information gap theory and the compression progress theory. The theories view curiosity as an intrinsic motivation to optimize for topological features of subgraphs induced by nodes visited in the environment. We use these proposed features as rewards for graph neural-network-based reinforcement learning. On multiple classes of synthetically generated graphs, we find that trained agents generalize to longer exploratory walks and larger environments than are seen during training. Our method computes more efficiently than the greedy evaluation of the relevant topological properties. The proposed intrinsic motivations bear particular relevance for recommender systems. We demonstrate that next-node recommendations considering curiosity are more predictive of human choices than PageRank centrality in several real-world graph environments.