Predicting the behavior of interacting humans by fusing data from multiple sources

TL;DR

This paper extends multi-fidelity methods to effectively combine limited high-fidelity human-in-the-loop data with abundant low-fidelity data, improving the modeling of human interactions.

Contribution

It introduces both model-based and model-free multi-fidelity approaches tailored for human-in-the-loop experiments, addressing generalization issues from online to real-world settings.

Findings

Model-based methods outperform in certain conditions

Model-free approaches offer robustness with limited high-fidelity data

Combined data sources improve predictive accuracy

Abstract

Multi-fidelity methods combine inexpensive low-fidelity simulations with costly but high-fidelity simulations to produce an accurate model of a system of interest at minimal cost. They have proven useful in modeling physical systems and have been applied to engineering problems such as wing-design optimization. During human-in-the-loop experimentation, it has become increasingly common to use online platforms, like Mechanical Turk, to run low-fidelity experiments to gather human performance data in an efficient manner. One concern with these experiments is that the results obtained from the online environment generalize poorly to the actual domain of interest. To address this limitation, we extend traditional multi-fidelity approaches to allow us to combine fewer data points from high-fidelity human-in-the-loop experiments with plentiful but less accurate data from low-fidelity experiments to produce accurate models of how humans interact. We present both model-based and model-free methods, and summarize the predictive performance of each method under different conditions.