Tipiano: Cascaded Piano Hand Motion Synthesis via Fingertip Priors

TL;DR

This paper introduces Tipiano, a four-stage hierarchical framework for synthesizing realistic piano hand motions by leveraging fingertip priors, achieving high accuracy and naturalness.

Contribution

The authors propose a novel hierarchical approach that combines statistical fingertip positioning with neural pose synthesis, along with providing expert-annotated fingerings for a new dataset.

Findings

Achieved F1 score of 0.910, outperforming diffusion baselines.

User study with 41 participants confirmed high motion quality.

Expert evaluation identified anticipatory motion as a key future improvement area.

Abstract

Synthesizing realistic piano hand motions requires both precision and naturalness. Physics-based methods achieve precision but produce stiff motions; data-driven models learn natural dynamics but struggle with positional accuracy. Piano motion exhibits a natural hierarchy: fingertip positions are nearly deterministic given piano geometry and fingering, while wrist and intermediate joints offer stylistic freedom. We present [OURS], a four-stage framework exploiting this hierarchy: (1) statistics-based fingertip positioning, (2) FiLM-conditioned trajectory refinement, (3) wrist estimation, and (4) STGCN-based pose synthesis. We contribute expert-annotated fingerings for the F\"urElise dataset (153 pieces, ~10 hours). Experiments demonstrate F1 = 0.910, substantially outperforming diffusion baselines (F1 = 0.121), with user study (N=41) confirming quality approaching motion capture. Expert evaluation by professional pianists (N=5) identified anticipatory motion as the key remaining gap, providing concrete directions for future improvement.