ROAM: a Rich Object Appearance Model with Application to Rotoscoping

TL;DR

This paper introduces ROAM, a comprehensive appearance model for rotoscoping that combines local and global features, enabling efficient, accurate, and adaptable object tracking in videos with minimal user input.

Contribution

ROAM presents a novel, rich appearance model that integrates local, global, and landmark features for improved rotoscoping, with efficient optimization and online adaptation capabilities.

Findings

Outperforms existing segmentation-based rotoscoping tools

Enables simple initialization and real-time adaptation

Provides both qualitative and quantitative validation

Abstract

Rotoscoping, the detailed delineation of scene elements through a video shot, is a painstaking task of tremendous importance in professional post-production pipelines. While pixel-wise segmentation techniques can help for this task, professional rotoscoping tools rely on parametric curves that offer the artists a much better interactive control on the definition, editing and manipulation of the segments of interest. Sticking to this prevalent rotoscoping paradigm, we propose a novel framework to capture and track the visual aspect of an arbitrary object in a scene, given a first closed outline of this object. This model combines a collection of local foreground/background appearance models spread along the outline, a global appearance model of the enclosed object and a set of distinctive foreground landmarks. The structure of this rich appearance model allows simple initialization, efficient iterative optimization with exact minimization at each step, and on-line adaptation in videos. We demonstrate qualitatively and quantitatively the merit of this framework through comparisons with tools based on either dynamic segmentation with a closed curve or pixel-wise binary labelling.