TouchWalker: Real-Time Avatar Locomotion from Touchscreen Finger Walking

TL;DR

TouchWalker is a real-time system that enables full-body avatar locomotion through finger-walking gestures on a touchscreen, providing continuous, context-aware motion including airborne phases, enhancing user immersion and control.

Contribution

It introduces a neural motion generator and touch interface that produce continuous, context-aware avatar locomotion from sparse finger input, surpassing prior gesture-based systems.

Findings

Improves user embodiment, enjoyment, and immersion.

Generates continuous full-body motion including airborne phases.

Outperforms baseline in user study across diverse tasks.

Abstract

We present TouchWalker, a real-time system for controlling full-body avatar locomotion using finger-walking gestures on a touchscreen. The system comprises two main components: TouchWalker-MotionNet, a neural motion generator that synthesizes full-body avatar motion on a per-frame basis from temporally sparse two-finger input, and TouchWalker-UI, a compact touch interface that interprets user touch input to avatar-relative foot positions. Unlike prior systems that rely on symbolic gesture triggers or predefined motion sequences, TouchWalker uses its neural component to generate continuous, context-aware full-body motion on a per-frame basis-including airborne phases such as running, even without input during mid-air steps-enabling more expressive and immediate interaction. To ensure accurate alignment between finger contacts and avatar motion, it employs a MoE-GRU architecture with a dedicated foot-alignment loss. We evaluate TouchWalker in a user study comparing it to a virtual joystick baseline with predefined motion across diverse locomotion tasks. Results show that TouchWalker improves users' sense of embodiment, enjoyment, and immersion.