Watch to Edit: Video Retargeting using Gaze
Kranthi Kumar, Moneish Kumar, Vineet Gandhi, Ramanathan Subramanian

TL;DR
This paper introduces a novel gaze-based video retargeting method that optimizes cropping, panning, and zooming to preserve salient content across different display aspect ratios, improving viewer experience.
Contribution
The authors propose a content-agnostic, gaze-driven video editing algorithm that re-edits entire movies with optimized cut, pan, and zoom operations using convex optimization techniques.
Findings
Significant improvement over state-of-the-art in computational efficiency
Enhanced qualitative viewing experience confirmed by user study
Effective re-editing of wide-angle static camera videos
Abstract
We present a novel approach to optimally retarget videos for varied displays with differing aspect ratios by preserving salient scene content discovered via eye tracking. Our algorithm performs editing with cut, pan and zoom operations by optimizing the path of a cropping window within the original video while seeking to (i) preserve salient regions, and (ii) adhere to the principles of cinematography. Our approach is (a) content agnostic as the same methodology is employed to re-edit a wide-angle video recording or a close-up movie sequence captured with a static or moving camera, and (b) independent of video length and can in principle re-edit an entire movie in one shot. Our algorithm consists of two steps. The first step employs gaze transition cues to detect time stamps where new cuts are to be introduced in the original video via dynamic programming. A subsequent step optimizes…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsVisual Attention and Saliency Detection · Advanced Image and Video Retrieval Techniques · Gaze Tracking and Assistive Technology
