SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning

TL;DR

SignSparK introduces a scalable, multilingual sign language production framework that uses sparse keyframes and advanced modeling to generate realistic, fluid signing sequences with precise editing capabilities.

Contribution

The paper presents a novel keyframe-based training paradigm and a large-scale CFM framework for efficient, high-fidelity multilingual sign language synthesis.

Findings

Achieves state-of-the-art results across multiple sign language benchmarks.

Enables high-fidelity, editable sign language generation in fewer than ten steps.

Scales to four different sign languages, establishing the largest multilingual SLP framework to date.

Abstract

Generating natural and linguistically accurate sign language avatars remains a formidable challenge. Current Sign Language Production (SLP) frameworks face a stark trade-off: direct text-to-pose models suffer from regression-to-the-mean effects, while dictionary-retrieval methods produce robotic, disjointed transitions. To resolve this, we propose a novel training paradigm that leverages sparse keyframes to capture the true underlying kinematic distribution of human signing. By predicting dense motion from these discrete anchors, our approach mitigates regression-to-the-mean while ensuring fluid articulation. To realize this paradigm at scale, we first introduce FAST, an ultra-efficient sign segmentation model that automatically mines precise temporal boundaries. We then present SignSparK, a large-scale Conditional Flow Matching (CFM) framework that utilizes these extracted anchors to synthesize 3D signing sequences in SMPL-X and MANO spaces. This keyframe-driven formulation also uniquely unlocks Keyframe-to-Pose (KF2P) generation, making precise spatiotemporal editing of signing sequences possible. Furthermore, our adopted reconstruction-based CFM objective also enables high-fidelity synthesis in fewer than ten sampling steps; this allows SignSparK to scale across four distinct sign languages, establishing the largest multilingual SLP framework to date. Finally, by integrating 3D Gaussian Splatting for photorealistic rendering, we demonstrate through extensive evaluation that SignSparK establishes a new state-of-the-art across diverse SLP tasks and multilingual benchmarks.