Adapting User Interfaces with Model-based Reinforcement Learning

TL;DR

This paper introduces a model-based reinforcement learning approach for adaptive user interfaces that plans and evaluates interface changes to optimize user experience while avoiding negative effects.

Contribution

It presents a novel conservative adaptation policy using predictive models to decide beneficial interface changes, improving over non-adaptive and frequency-based methods.

Findings

Outperforms non-adaptive policies in empirical tests

Outperforms frequency-based policies in simulations

Effective in adaptive menu scenarios

Abstract

Adapting an interface requires taking into account both the positive and negative effects that changes may have on the user. A carelessly picked adaptation may impose high costs to the user -- for example, due to surprise or relearning effort -- or "trap" the process to a suboptimal design immaturely. However, effects on users are hard to predict as they depend on factors that are latent and evolve over the course of interaction. We propose a novel approach for adaptive user interfaces that yields a conservative adaptation policy: It finds beneficial changes when there are such and avoids changes when there are none. Our model-based reinforcement learning method plans sequences of adaptations and consults predictive HCI models to estimate their effects. We present empirical and simulation results from the case of adaptive menus, showing that the method outperforms both a non-adaptive and a frequency-based policy.