2nd Place Solution for PVUW Challenge 2024: Video Panoptic Segmentation

TL;DR

This paper presents a robust integrated solution for video panoptic segmentation that combines baseline mask generation with an additional semantic segmentation model, achieving state-of-the-art results in the PVUW Challenge 2024.

Contribution

The authors enhance existing VPS methods by integrating an additional semantic segmentation model, significantly improving performance on the PVUW Challenge.

Findings

Achieved VPQ scores of 56.36 (dev) and 57.12 (test)

Ranked 2nd in the PVUW Challenge 2024

Demonstrated the effectiveness of combining baseline and semantic segmentation models

Abstract

Video Panoptic Segmentation (VPS) is a challenging task that is extends from image panoptic segmentation.VPS aims to simultaneously classify, track, segment all objects in a video, including both things and stuff. Due to its wide application in many downstream tasks such as video understanding, video editing, and autonomous driving. In order to deal with the task of video panoptic segmentation in the wild, we propose a robust integrated video panoptic segmentation solution. We use DVIS++ framework as our baseline to generate the initial masks. Then,we add an additional image semantic segmentation model to further improve the performance of semantic classes.Finally, our method achieves state-of-the-art performance with a VPQ score of 56.36 and 57.12 in the development and test phases, respectively, and ultimately ranked 2nd in the VPS track of the PVUW Challenge at CVPR2024.