Animation-by-Demonstration Computer Puppetry Authoring Framework

TL;DR

The paper introduces MOP, a real-time animation-by-demonstration framework that uses HMMs to control virtual character motions based on user actions, enabling intuitive puppetry without complex motion synthesis.

Contribution

It presents a novel real-time puppetry control system using HMMs to learn and predict user actions for virtual character animation.

Findings

Real-time recognition of user actions during puppetry.

Effective control of virtual character motions based on demonstrations.

Demonstrated flexibility and efficiency in various scenarios.

Abstract

This paper presents Master of Puppets (MOP), an animation-by-demonstration framework that allows users to control the motion of virtual characters (puppets) in real time. In the first step, the user is asked to perform the necessary actions that correspond to the character's motions. The user's actions are recorded, and a hidden Markov model (HMM) is used to learn the temporal profile of the actions. During the runtime of the framework, the user controls the motions of the virtual character based on the specified activities. The advantage of the MOP framework is that it recognizes and follows the progress of the user's actions in real time. Based on the forward algorithm, the method predicts the evolution of the user's actions, which corresponds to the evolution of the character's motion. This method treats characters as puppets that can perform only one motion at a time. This means that combinations of motion segments (motion synthesis), as well as the interpolation of individual motion sequences, are not provided as functionalities. By implementing the framework and presenting several computer puppetry scenarios, its efficiency and flexibility in animating virtual characters is demonstrated.