GAZED- Gaze-guided Cinematic Editing of Wide-Angle Monocular Video Recordings

TL;DR

GAZED is a novel method that uses eye-gaze data to automatically edit wide-angle monocular videos, producing cinematic and aesthetically pleasing narratives by modeling shot selection as an energy minimization problem.

Contribution

This work introduces a gaze-guided editing framework that leverages eye-tracking to automate cinematic video editing from static wide-angle recordings, incorporating cinematographic constraints.

Findings

GAZED outperforms competing methods in user studies.

Eye-gaze effectively guides shot selection for cinematic quality.

The method produces vivid, narrative-aligned edited videos.

Abstract

We present GAZED- eye GAZe-guided EDiting for videos captured by a solitary, static, wide-angle and high-resolution camera. Eye-gaze has been effectively employed in computational applications as a cue to capture interesting scene content; we employ gaze as a proxy to select shots for inclusion in the edited video. Given the original video, scene content and user eye-gaze tracks are combined to generate an edited video comprising cinematically valid actor shots and shot transitions to generate an aesthetic and vivid representation of the original narrative. We model cinematic video editing as an energy minimization problem over shot selection, whose constraints capture cinematographic editing conventions. Gazed scene locations primarily determine the shots constituting the edited video. Effectiveness of GAZED against multiple competing methods is demonstrated via a psychophysical study involving 12 users and twelve performance videos.