Adversarial construction as a potential solution to the experiment design problem in large task spaces

TL;DR

This paper introduces an adversarial construction method to efficiently explore large task spaces, specifically in binary sequence prediction tasks modeled by hidden Markov models, to identify tasks that provoke novel behaviors.

Contribution

It proposes a novel adversarial approach for experimental design in high-dimensional task spaces, improving over random sampling for discovering diverse behaviors.

Findings

Adversarial construction outperforms random sampling in identifying novel behaviors.

The method effectively navigates large task spaces to find qualitatively different tasks.

Results suggest potential for improved experimental design in complex domains.

Abstract

Despite decades of work, we still lack a robust, task-general theory of human behavior even in the simplest domains. In this paper we tackle the generality problem head-on, by aiming to develop a unified model for all tasks embedded in a task-space. In particular we consider the space of binary sequence prediction tasks where the observations are generated by the space parameterized by hidden Markov models (HMM). As the space of tasks is large, experimental exploration of the entire space is infeasible. To solve this problem we propose the adversarial construction approach, which helps identify tasks that are most likely to elicit a qualitatively novel behavior. Our results suggest that adversarial construction significantly outperforms random sampling of environments and therefore could be used as a proxy for optimal experimental design in high-dimensional task spaces.