Latest Object Memory Management for Temporally Consistent Video Instance Segmentation

TL;DR

This paper introduces Latest Object Memory Management (LOMM), a novel approach for temporally consistent video instance segmentation that enhances long-term object tracking by explicitly modeling object presence and employing a decoupled association strategy.

Contribution

The paper proposes LOMM and Decoupled Object Association (DOA), novel methods that improve long-term tracking accuracy and identity consistency in video instance segmentation.

Findings

Achieves state-of-the-art AP score of 54.0 on YouTube-VIS 2022.

Significantly improves long-term object tracking and identity management.

Demonstrates robustness in dynamic scenes with frequent object appearance and disappearance.

Abstract

In this paper, we present Latest Object Memory Management (LOMM) for temporally consistent video instance segmentation that significantly improves long-term instance tracking. At the core of our method is Latest Object Memory (LOM), which robustly tracks and continuously updates the latest states of objects by explicitly modeling their presence in each frame. This enables consistent tracking and accurate identity management across frames, enhancing both performance and reliability through the VIS process. Moreover, we introduce Decoupled Object Association (DOA), a strategy that separately handles newly appearing and already existing objects. By leveraging our memory system, DOA accurately assigns object indices, improving matching accuracy and ensuring stable identity consistency, even in dynamic scenes where objects frequently appear and disappear. Extensive experiments and ablation studies demonstrate the superiority of our method over traditional approaches, setting a new benchmark in VIS. Notably, our LOMM achieves state-of-the-art AP score of 54.0 on YouTube-VIS 2022, a dataset known for its challenging long videos. Project page: https://seung-hun-lee.github.io/projects/LOMM/