# Towards Segmenting Anything That Moves

## TL;DR

This paper introduces a learning-based method for segmenting moving objects in videos by combining motion cues from optical flow with appearance features, outperforming prior approaches on multiple benchmarks.

## Contribution

It presents a novel spatio-temporal grouping approach that leverages motion and appearance cues, and introduces new benchmarks for generic moving object detection.

## Key findings

- Outperforms all prior work on FBMS dataset
- Matches top-down methods on common categories
- Significantly outperforms on unseen categories

## Abstract

Detecting and segmenting individual objects, regardless of their category, is crucial for many applications such as action detection or robotic interaction. While this problem has been well-studied under the classic formulation of spatio-temporal grouping, state-of-the-art approaches do not make use of learning-based methods. To bridge this gap, we propose a simple learning-based approach for spatio-temporal grouping. Our approach leverages motion cues from optical flow as a bottom-up signal for separating objects from each other. Motion cues are then combined with appearance cues that provide a generic objectness prior for capturing the full extent of objects. We show that our approach outperforms all prior work on the benchmark FBMS dataset. One potential worry with learning-based methods is that they might overfit to the particular type of objects that they have been trained on. To address this concern, we propose two new benchmarks for generic, moving object detection, and show that our model matches top-down methods on common categories, while significantly out-performing both top-down and bottom-up methods on never-before-seen categories.