Demystifying Reward Design in Reinforcement Learning for Upper Extremity Interaction: Practical Guidelines for Biomechanical Simulations in HCI

TL;DR

This paper provides practical guidelines for designing reward functions in reinforcement learning for biomechanical simulations in HCI, emphasizing the roles of proximity incentives, completion bonuses, and effort terms to improve efficiency and effectiveness.

Contribution

It systematically analyzes reward component impacts and offers validated guidelines to simplify reward design for biomechanical HCI simulations without requiring RL expertise.

Findings

Proximity incentives are crucial for guiding movement.

Completion bonuses ensure task success.

Effort terms refine motion regularity when scaled properly.

Abstract

Designing effective reward functions is critical for reinforcement learning-based biomechanical simulations, yet HCI researchers and practitioners often waste (computation) time with unintuitive trial-and-error tuning. This paper demystifies reward function design by systematically analyzing the impact of effort minimization, task completion bonuses, and target proximity incentives on typical HCI tasks such as pointing, tracking, and choice reaction. We show that proximity incentives are essential for guiding movement, while completion bonuses ensure task success. Effort terms, though optional, help refine motion regularity when appropriately scaled. We perform an extensive analysis of how sensitive task success and completion time depend on the weights of these three reward components. From these results we derive practical guidelines to create plausible biomechanical simulations without the need for reinforcement learning expertise, which we then validate on remote control and keyboard typing tasks. This paper advances simulation-based interaction design and evaluation in HCI by improving the efficiency and applicability of biomechanical user modeling for real-world interface development.