OMG-Seg: Is One Model Good Enough For All Segmentation?

TL;DR

OMG-Seg is a unified transformer-based model capable of handling a wide range of segmentation tasks, including image, video, open vocabulary, and interactive segmentation, with reduced computational costs.

Contribution

This work introduces OMG-Seg, the first unified model that efficiently addresses multiple segmentation tasks with a single architecture.

Findings

Supports over ten segmentation tasks simultaneously

Reduces computational and parameter overhead

Achieves satisfactory performance across diverse tasks

Abstract

In this work, we address various segmentation tasks, each traditionally tackled by distinct or partially unified models. We propose OMG-Seg, One Model that is Good enough to efficiently and effectively handle all the segmentation tasks, including image semantic, instance, and panoptic segmentation, as well as their video counterparts, open vocabulary settings, prompt-driven, interactive segmentation like SAM, and video object segmentation. To our knowledge, this is the first model to handle all these tasks in one model and achieve satisfactory performance. We show that OMG-Seg, a transformer-based encoder-decoder architecture with task-specific queries and outputs, can support over ten distinct segmentation tasks and yet significantly reduce computational and parameter overhead across various tasks and datasets. We rigorously evaluate the inter-task influences and correlations during co-training. Code and models are available at https://github.com/lxtGH/OMG-Seg.