Predicting the behavior of interacting humans by fusing data from multiple sources

TL;DR

This paper extends multi-fidelity modeling techniques to combine limited high-fidelity human interaction data with abundant low-fidelity data, improving the accuracy of human behavior predictions in online experiments.

Contribution

It introduces novel model-based and model-free multi-fidelity methods specifically designed for human-in-the-loop data fusion.

Findings

Model-based methods outperform model-free in predictive accuracy.

Combining high- and low-fidelity data reduces the need for extensive high-fidelity experiments.

The approaches improve generalization of human behavior models from online data.

Abstract

Multi-fidelity methods combine inexpensive low-fidelity simulations with costly but highfidelity simulations to produce an accurate model of a system of interest at minimal cost. They have proven useful in modeling physical systems and have been applied to engineering problems such as wing-design optimization. During human-in-the-loop experimentation, it has become increasingly common to use online platforms, like Mechanical Turk, to run low-fidelity experiments to gather human performance data in an efficient manner. One concern with these experiments is that the results obtained from the online environment generalize poorly to the actual domain of interest. To address this limitation, we extend traditional multi-fidelity approaches to allow us to combine fewer data points from high-fidelity human-in-the-loop experiments with plentiful but less accurate data from low-fidelity experiments to produce accurate models of how humans interact. We present both model-based and model-free methods, and summarize the predictive performance of each method under dierent conditions.