Empirically Verifying Hypotheses Using Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning framework for hypothesis verification, enabling agents to actively generate observations to test hypotheses about world dynamics, with success demonstrated through structured and fine-tuned policies.

Contribution

It formulates hypothesis verification as an RL problem and leverages hypothesis structure for effective agent training, advancing the capabilities of RL in scientific hypothesis testing.

Findings

RL agents can verify hypotheses with structured approaches

Factorization of hypotheses improves training success

Fine-tuning enables verification of complex hypotheses

Abstract

This paper formulates hypothesis verification as an RL problem. Specifically, we aim to build an agent that, given a hypothesis about the dynamics of the world, can take actions to generate observations which can help predict whether the hypothesis is true or false. Existing RL algorithms fail to solve this task, even for simple environments. In order to train the agents, we exploit the underlying structure of many hypotheses, factorizing them as {pre-condition, action sequence, post-condition} triplets. By leveraging this structure we show that RL agents are able to succeed at the task. Furthermore, subsequent fine-tuning of the policies allows the agent to correctly verify hypotheses not amenable to the above factorization.