Optimal Behavior Prior: Data-Efficient Human Models for Improved Human-AI Collaboration

TL;DR

This paper introduces a simple yet effective prior based on near-optimal human behavior to create data-efficient models that improve human-AI collaboration and generalize well to new environments.

Contribution

It proposes using optimal behavior as a prior for human models, significantly enhancing data efficiency and generalization in human-AI collaboration tasks.

Findings

Models with the optimal behavior prior are more data-efficient.

Prior-based models generalize better to new environments.

Improved human-AI collaboration performance.

Abstract

AI agents designed to collaborate with people benefit from models that enable them to anticipate human behavior. However, realistic models tend to require vast amounts of human data, which is often hard to collect. A good prior or initialization could make for more data-efficient training, but what makes for a good prior on human behavior? Our work leverages a very simple assumption: people generally act closer to optimal than to random chance. We show that using optimal behavior as a prior for human models makes these models vastly more data-efficient and able to generalize to new environments. Our intuition is that such a prior enables the training to focus one's precious real-world data on capturing the subtle nuances of human suboptimality, instead of on the basics of how to do the task in the first place. We also show that using these improved human models often leads to better human-AI collaboration performance compared to using models based on real human data alone.