HYouTube: Video Harmonization Dataset

TL;DR

This paper introduces HYouTube, a new dataset for video harmonization, addressing the lack of public datasets and facilitating research on adjusting foregrounds in composite videos for better visual consistency.

Contribution

The paper presents the first large-scale video harmonization dataset, HYouTube, including synthetic and real composite videos, to advance research in this underexplored area.

Findings

Dataset enables training of video harmonization models.

Synthetic and real composite videos highlight domain gaps.

Provides a benchmark for future video harmonization research.

Abstract

Video composition aims to generate a composite video by combining the foreground of one video with the background of another video, but the inserted foreground may be incompatible with the background in terms of color and illumination. Video harmonization aims to adjust the foreground of a composite video to make it compatible with the background. So far, video harmonization has only received limited attention and there is no public dataset for video harmonization. In this work, we construct a new video harmonization dataset HYouTube by adjusting the foreground of real videos to create synthetic composite videos. Considering the domain gap between real composite videos and synthetic composite videos, we additionally create 100 real composite videos via copy-and-paste. Datasets are available at https://github.com/bcmi/Video-Harmonization-Dataset-HYouTube.