Video Region Annotation with Sparse Bounding Boxes

TL;DR

This paper introduces a novel method using a Volumetric Graph Convolutional Network to automatically generate detailed region boundaries in videos from sparse bounding box annotations, reducing the need for dense labeling.

Contribution

It presents a new approach leveraging VGCN for boundary prediction from sparse annotations, improving accuracy and generalization over existing methods.

Findings

Effective boundary generation demonstrated on real and synthetic datasets

Outperforms existing solutions in accuracy and robustness

Ablation studies confirm the importance of spatio-temporal information

Abstract

Video analysis has been moving towards more detailed interpretation (e.g. segmentation) with encouraging progresses. These tasks, however, increasingly rely on densely annotated training data both in space and time. Since such annotation is labour-intensive, few densely annotated video data with detailed region boundaries exist. This work aims to resolve this dilemma by learning to automatically generate region boundaries for all frames of a video from sparsely annotated bounding boxes of target regions. We achieve this with a Volumetric Graph Convolutional Network (VGCN), which learns to iteratively find keypoints on the region boundaries using the spatio-temporal volume of surrounding appearance and motion. The global optimization of VGCN makes it significantly stronger and generalize better than existing solutions. Experimental results using two latest datasets (one real and one synthetic), including ablation studies, demonstrate the effectiveness and superiority of our method.