Increasing Interaction Fidelity: Training Routines for Biomechanical Models in HCI

TL;DR

This paper introduces improved training routines for biomechanical models in HCI, enhancing interaction fidelity and enabling more complex models for realistic human-like movements in touchscreen tasks.

Contribution

It presents practical training improvements like curriculum learning and action masking that significantly boost interaction fidelity in biomechanical models for HCI.

Findings

Training routines reduce time required for learning.

Interaction fidelity surpasses previous methods.

Complex models become feasible for realistic touch behavior.

Abstract

Biomechanical forward simulation holds great potential for HCI, enabling the generation of human-like movements in interactive tasks. However, training biomechanical models with reinforcement learning is challenging, particularly for precise and dexterous movements like those required for touchscreen interactions on mobile devices. Current approaches are limited in their interaction fidelity, require restricting the underlying biomechanical model to reduce complexity, and do not generalize well. In this work, we propose practical improvements to training routines that reduce training time, increase interaction fidelity beyond existing methods, and enable the use of more complex biomechanical models. Using a touchscreen pointing task, we demonstrate that curriculum learning, action masking, more complex network configurations, and simple adjustments to the simulation environment can significantly improve the agent's ability to learn accurate touch behavior. Our work provides HCI researchers with practical tips and training routines for developing better biomechanical models of human-like interaction fidelity.