Visualizing the Invisible: Occluded Vehicle Segmentation and Recovery

TL;DR

This paper introduces an iterative multi-task framework for segmenting and recovering occluded vehicles, utilizing adversarial training and a shared network to improve accuracy in both synthetic and real-world images.

Contribution

It presents a novel multi-task approach with coupled discriminators and an auxiliary 3D model pool for enhanced occluded vehicle segmentation and appearance recovery.

Findings

Outperforms state-of-the-art in occluded vehicle segmentation and appearance recovery.

The method improves occluded vehicle tracking in real-world videos.

Introduces the Occluded Vehicle dataset for evaluation.

Abstract

In this paper, we propose a novel iterative multi-task framework to complete the segmentation mask of an occluded vehicle and recover the appearance of its invisible parts. In particular, to improve the quality of the segmentation completion, we present two coupled discriminators and introduce an auxiliary 3D model pool for sampling authentic silhouettes as adversarial samples. In addition, we propose a two-path structure with a shared network to enhance the appearance recovery capability. By iteratively performing the segmentation completion and the appearance recovery, the results will be progressively refined. To evaluate our method, we present a dataset, the Occluded Vehicle dataset, containing synthetic and real-world occluded vehicle images. We conduct comparison experiments on this dataset and demonstrate that our model outperforms the state-of-the-art in tasks of recovering segmentation mask and appearance for occluded vehicles. Moreover, we also demonstrate that our appearance recovery approach can benefit the occluded vehicle tracking in real-world videos.