MatAnyone: Stable Video Matting with Consistent Memory Propagation

TL;DR

MatAnyone is a new video matting framework that uses memory propagation and a large dataset to improve stability and accuracy in complex scenes, outperforming existing methods.

Contribution

It introduces a memory-based approach with region-adaptive fusion, a new dataset, and a training strategy that enhances video matting performance.

Findings

Outperforms existing methods in diverse scenarios.

Provides stable and accurate matting results.

Utilizes a novel memory propagation module.

Abstract

Auxiliary-free human video matting methods, which rely solely on input frames, often struggle with complex or ambiguous backgrounds. To address this, we propose MatAnyone, a robust framework tailored for target-assigned video matting. Specifically, building on a memory-based paradigm, we introduce a consistent memory propagation module via region-adaptive memory fusion, which adaptively integrates memory from the previous frame. This ensures semantic stability in core regions while preserving fine-grained details along object boundaries. For robust training, we present a larger, high-quality, and diverse dataset for video matting. Additionally, we incorporate a novel training strategy that efficiently leverages large-scale segmentation data, boosting matting stability. With this new network design, dataset, and training strategy, MatAnyone delivers robust and accurate video matting results in diverse real-world scenarios, outperforming existing methods.