Interpretable Models for Understanding Immersive Simulations

TL;DR

This paper develops methods to evaluate and compare the interpretability of models analyzing high-dimensional time series data from immersive simulations, emphasizing human-aligned representations and Bayesian approaches.

Contribution

It introduces interpretability tests for model evaluation, showing that models optimized for interpretability differ from those optimized for statistical criteria, and highlights the effectiveness of Bayesian models.

Findings

Models optimized for interpretability differ from statistical criteria models.

Bayesian models perform well on both interpretability and statistical measures.

Interpretability tests align model evaluation with human expectations.

Abstract

This paper describes methods for comparative evaluation of the interpretability of models of high dimensional time series data inferred by unsupervised machine learning algorithms. The time series data used in this investigation were logs from an immersive simulation like those commonly used in education and healthcare training. The structures learnt by the models provide representations of participants' activities in the simulation which are intended to be meaningful to people's interpretation. To choose the model that induces the best representation, we designed two interpretability tests, each of which evaluates the extent to which a model's output aligns with people's expectations or intuitions of what has occurred in the simulation. We compared the performance of the models on these interpretability tests to their performance on statistical information criteria. We show that the models that optimize interpretability quality differ from those that optimize (statistical) information theoretic criteria. Furthermore, we found that a model using a fully Bayesian approach performed well on both the statistical and human-interpretability measures. The Bayesian approach is a good candidate for fully automated model selection, i.e., when direct empirical investigations of interpretability are costly or infeasible.