Towards a Deep(er) Understanding of Interaction through Modeling, Simulation, and Optimization

TL;DR

This paper advocates augmenting traditional user-centered design with simulation and optimization to better understand, develop, and validate human-computer interaction techniques, exemplified through acoustic levitation.

Contribution

It introduces an optimization-based framework for modeling, simulating, and improving HCI, advancing the creation of adaptive and intelligent user interfaces.

Findings

Successful virtual development of acoustic levitation interaction techniques

Framework supports validation of HCI theories

Enhances understanding of interaction phenomena

Abstract

The traditional user-centered design process can hardly keep up with the ever faster technical development and increasingly diverse user preferences. As a solution, we propose to augment the tried-and-tested approach of conducting user studies with simulation and optimization of the entire human-computer interaction loop. This approach allows to better understand phenomena through explicit modeling, build virtual prototypes through simulation, and improve interaction techniques through optimization. Building predictive user models also supports the creation and validation of HCI theories, and constitutes a decisive step towards new, intelligent, and adaptive user interfaces. We report our experience in virtually developing new interaction techniques on the example of acoustic levitation, and present our optimization-based framework for HCI. With this, we strive to gain a better understanding of interaction and at the same time feed the discussion on questions such as which tools and tutorials are necessary to make virtual prototyping more accessible to different audiences.