Collaborative Attention Memory Network for Video Object Segmentation

TL;DR

This paper introduces a collaborative attention memory network with an enhanced segmentation head for semi-supervised video object segmentation, addressing issues like false predictions and occlusion, and achieves competitive results on Youtube-VOS.

Contribution

It proposes a novel object context scheme and a segmentation head with Feature Pyramid Attention, combined with an ensemble network to improve segmentation accuracy.

Findings

Achieved 6th place on Youtube-VOS with 83.5% score.

Enhanced object information gathering reduces false predictions.

Ensemble of networks improves overall segmentation performance.

Abstract

Semi-supervised video object segmentation is a fundamental yet Challenging task in computer vision. Embedding matching based CFBI series networks have achieved promising results by foreground-background integration approach. Despite its superior performance, these works exhibit distinct shortcomings, especially the false predictions caused by little appearance instances in first frame, even they could easily be recognized by previous frame. Moreover, they suffer from object's occlusion and error drifts. In order to overcome the shortcomings , we propose Collaborative Attention Memory Network with an enhanced segmentation head. We introduce a object context scheme that explicitly enhances the object information, which aims at only gathering the pixels that belong to the same category as a given pixel as its context. Additionally, a segmentation head with Feature Pyramid Attention(FPA) module is adopted to perform spatial pyramid attention structure on high-level output. Furthermore, we propose an ensemble network to combine STM network with all these new refined CFBI network. Finally, we evaluated our approach on the 2021 Youtube-VOS challenge where we obtain 6th place with an overall score of 83.5\%.