FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation

TL;DR

FastInst is a simple, query-based, real-time instance segmentation model that achieves high accuracy and speed, outperforming many existing methods without complex techniques.

Contribution

The paper introduces FastInst, a novel, efficient query-based framework for real-time instance segmentation that balances speed and accuracy with innovative design strategies.

Findings

FastInst runs at 32.5 FPS on COCO test-dev.

Achieves 40.5 AP, surpassing many real-time models.

Outperforms state-of-the-art real-time counterparts in speed and accuracy.

Abstract

Recent attention in instance segmentation has focused on query-based models. Despite being non-maximum suppression (NMS)-free and end-to-end, the superiority of these models on high-accuracy real-time benchmarks has not been well demonstrated. In this paper, we show the strong potential of query-based models on efficient instance segmentation algorithm designs. We present FastInst, a simple, effective query-based framework for real-time instance segmentation. FastInst can execute at a real-time speed (i.e., 32.5 FPS) while yielding an AP of more than 40 (i.e., 40.5 AP) on COCO test-dev without bells and whistles. Specifically, FastInst follows the meta-architecture of recently introduced Mask2Former. Its key designs include instance activation-guided queries, dual-path update strategy, and ground truth mask-guided learning, which enable us to use lighter pixel decoders, fewer Transformer decoder layers, while achieving better performance. The experiments show that FastInst outperforms most state-of-the-art real-time counterparts, including strong fully convolutional baselines, in both speed and accuracy. Code can be found at https://github.com/junjiehe96/FastInst .