Mind & Motion: Opportunities and Applications of Integrating Biomechanics and Cognitive Models in HCI

TL;DR

This paper discusses integrating cognitive and biomechanical models to enhance user interaction simulations, aiming to improve interface design, testing, and personalization in HCI.

Contribution

It introduces the concept of connecting cognition and biomechanics models to create comprehensive, embodied user simulations for better HCI analysis and system adaptation.

Findings

Potential for more accurate prediction of user behavior and physical effort.

Applications in UI/UX design, automated testing, and personalized interfaces.

Identification of key challenges and future research directions.

Abstract

Computational models of how users perceive and act within a virtual or physical environment offer enormous potential for the understanding and design of user interactions. Cognition models have been used to understand the role of attention and individual preferences and beliefs on human decision making during interaction, while biomechanical simulations have been successfully applied to analyse and predict physical effort, fatigue, and discomfort. The next frontier in HCI lies in connecting these models to enable robust, diverse, and representative simulations of different user groups. These embodied user simulations could predict user intents, strategies, and movements during interaction more accurately, benchmark interfaces and interaction techniques in terms of performance and ergonomics, and guide adaptive system design. This UIST workshop explores ideas for integrating computational models into HCI and discusses use cases such as UI/UX design, automated system testing, and personalised adaptive interfaces. It brings researchers from relevant disciplines together to identify key opportunities and challenges as well as feasible next steps for bridging mind and motion to simulate interactive user behaviour.