# RANet: Ranking Attention Network for Fast Video Object Segmentation

## TL;DR

RANet is a real-time, highly accurate video object segmentation network that combines matching and propagation techniques using a novel ranking attention module, achieving state-of-the-art speed and accuracy.

## Contribution

The paper introduces a ranking attention module within an encoder-decoder framework for end-to-end pixel-level similarity learning in VOS.

## Key findings

- Achieves 85.5% J&F on DAVIS-16 at 33ms per frame
- Outperforms state-of-the-art VOS methods with online learning
- Balances speed and accuracy effectively

## Abstract

Despite online learning (OL) techniques have boosted the performance of semi-supervised video object segmentation (VOS) methods, the huge time costs of OL greatly restrict their practicality. Matching based and propagation based methods run at a faster speed by avoiding OL techniques. However, they are limited by sub-optimal accuracy, due to mismatching and drifting problems. In this paper, we develop a real-time yet very accurate Ranking Attention Network (RANet) for VOS. Specifically, to integrate the insights of matching based and propagation based methods, we employ an encoder-decoder framework to learn pixel-level similarity and segmentation in an end-to-end manner. To better utilize the similarity maps, we propose a novel ranking attention module, which automatically ranks and selects these maps for fine-grained VOS performance. Experiments on DAVIS-16 and DAVIS-17 datasets show that our RANet achieves the best speed-accuracy trade-off, e.g., with 33 milliseconds per frame and J&F=85.5% on DAVIS-16. With OL, our RANet reaches J&F=87.1% on DAVIS-16, exceeding state-of-the-art VOS methods. The code can be found at https://github.com/Storife/RANet.