Universal Instance Perception as Object Discovery and Retrieval

TL;DR

UNINEXT is a unified, flexible model for diverse instance perception tasks that improves data efficiency and performance across multiple benchmarks by reformulating tasks into object discovery and retrieval.

Contribution

The paper introduces UNINEXT, a universal model that unifies various instance perception tasks into a single framework, enabling joint training and efficient multi-task handling.

Findings

Outperforms on 20 benchmarks across 10 tasks

Unified model reduces redundant computation

Effective in low-data scenarios for certain tasks

Abstract

All instance perception tasks aim at finding certain objects specified by some queries such as category names, language expressions, and target annotations, but this complete field has been split into multiple independent subtasks. In this work, we present a universal instance perception model of the next generation, termed UNINEXT. UNINEXT reformulates diverse instance perception tasks into a unified object discovery and retrieval paradigm and can flexibly perceive different types of objects by simply changing the input prompts. This unified formulation brings the following benefits: (1) enormous data from different tasks and label vocabularies can be exploited for jointly training general instance-level representations, which is especially beneficial for tasks lacking in training data. (2) the unified model is parameter-efficient and can save redundant computation when handling multiple tasks simultaneously. UNINEXT shows superior performance on 20 challenging benchmarks from 10 instance-level tasks including classical image-level tasks (object detection and instance segmentation), vision-and-language tasks (referring expression comprehension and segmentation), and six video-level object tracking tasks. Code is available at https://github.com/MasterBin-IIAU/UNINEXT.