SDI-Paste: Synthetic Dynamic Instance Copy-Paste for Video Instance Segmentation
Sahir Shrestha, Weihao Li, Gao Zhu, Nick Barnes

TL;DR
SDI-Paste introduces a scalable video data augmentation pipeline that synthetically generates and incorporates dynamic object instances into videos, significantly improving performance in video instance segmentation tasks.
Contribution
The paper presents a novel, scalable method for synthetic dynamic object augmentation in videos specifically designed for video instance segmentation.
Findings
Achieved +2.9 AP (6.5%) improvement on Youtube-VIS 2021.
Achieved +2.1 AP (4.9%) improvement on Youtube-VIS 2021.
Demonstrated effectiveness of synthetic dynamic augmentation in video segmentation.
Abstract
Data augmentation methods such as Copy-Paste have been studied as effective ways to expand training datasets while incurring minimal costs. While such methods have been extensively implemented for image level tasks, we found no scalable implementation of Copy-Paste built specifically for video tasks. In this paper, we leverage the recent growth in video fidelity of generative models to explore effective ways of incorporating synthetically generated objects into existing video datasets to artificially expand object instance pools. We first procure synthetic video sequences featuring objects that morph dynamically with time. Our carefully devised pipeline automatically segments then copy-pastes these dynamic instances across the frames of any target background video sequence. We name our video data augmentation pipeline Synthetic Dynamic Instance Copy-Paste, and test it on the complex…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsVideo Analysis and Summarization · Advanced Image and Video Retrieval Techniques · Generative Adversarial Networks and Image Synthesis
Methodssimple Copy-Paste
