RealityEffects: Augmenting 3D Volumetric Videos with Object-Centric Annotation and Dynamic Visual Effects

TL;DR

RealityEffects is a novel desktop tool that allows users to easily add and animate object-centric visual effects in 3D volumetric videos, improving editing efficiency and viewer engagement.

Contribution

The paper presents a new taxonomy for object-centric augmentations and a responsive editing interface for volumetric videos, advancing the state of interactive 3D video editing.

Findings

User study shows reduced editing barriers.

Enhanced viewer engagement with dynamic effects.

Effective taxonomy for object-centric augmentations.

Abstract

This paper introduces RealityEffects, a desktop authoring interface designed for editing and augmenting 3D volumetric videos with object-centric annotations and visual effects. RealityEffects enhances volumetric capture by introducing a novel method for augmenting captured physical motion with embedded, responsive visual effects, referred to as object-centric augmentation. In RealityEffects, users can interactively attach various visual effects to physical objects within the captured 3D scene, enabling these effects to dynamically move and animate in sync with the corresponding physical motion and body movements. The primary contribution of this paper is the development of a taxonomy for such object-centric augmentations, which includes annotated labels, highlighted objects, ghost effects, and trajectory visualization. This taxonomy is informed by an analysis of 120 edited videos featuring object-centric visual effects. The findings from our user study confirm that our direct manipulation techniques lower the barriers to editing and annotating volumetric captures, thereby enhancing interactive and engaging viewing experiences of 3D volumetric videos.