Parallel Pre-trained Transformers (PPT) for Synthetic Data-based Instance Segmentation

TL;DR

This paper introduces a Parallel Pre-trained Transformers framework for synthetic data-based instance segmentation, leveraging pre-trained vision transformers and parallel feature learning to improve generalization and robustness.

Contribution

It proposes a novel PPT framework that combines multiple pre-trained transformers with parallel feature learning and fusion for synthetic data segmentation.

Findings

Achieved 65.155% mAP on CVPR2022 AVA Challenge

Utilized parallel transformers for improved robustness

Fused multiple models for better segmentation results

Abstract

Recently, Synthetic data-based Instance Segmentation has become an exceedingly favorable optimization paradigm since it leverages simulation rendering and physics to generate high-quality image-annotation pairs. In this paper, we propose a Parallel Pre-trained Transformers (PPT) framework to accomplish the synthetic data-based Instance Segmentation task. Specifically, we leverage the off-the-shelf pre-trained vision Transformers to alleviate the gap between natural and synthetic data, which helps to provide good generalization in the downstream synthetic data scene with few samples. Swin-B-based CBNet V2, SwinL-based CBNet V2 and Swin-L-based Uniformer are employed for parallel feature learning, and the results of these three models are fused by pixel-level Non-maximum Suppression (NMS) algorithm to obtain more robust results. The experimental results reveal that PPT ranks first in the CVPR2022 AVA Accessibility Vision and Autonomy Challenge, with a 65.155% mAP.