Tiny-YOLOSAM: Fast Hybrid Image Segmentation

TL;DR

Tiny-YOLOSAM combines a fast object detector with a lightweight segmentation model to achieve high-quality, efficient image segmentation suitable for real-time applications, significantly reducing processing time while maintaining strong segmentation coverage.

Contribution

The paper introduces Tiny-YOLOSAM, a hybrid pipeline that leverages YOLOv12 for object detection to guide segmentation, improving speed and coverage over existing lightweight segmentation methods.

Findings

Substantially improves class-agnostic coverage (AR from 16.4% to 77.1%)

Reduces runtime from 49.20s to 10.39s per image (4.7x faster)

Maintains high segmentation quality with minimal prompts

Abstract

The Segment Anything Model (SAM) enables promptable, high-quality segmentation but is often too computationally expensive for latency-critical settings. TinySAM is a lightweight, distilled SAM variant that preserves strong zero-shot mask quality, yet its "segment-everything" mode still requires hundreds of prompts and remains slow in practice. We first replicate TinySAM on COCO val2017 using official checkpoints, matching the reported AP within 0.03%, establishing a reliable experimental baseline. Building on this, we propose Tiny-YOLOSAM, a fast hybrid pipeline that uses a recent YOLO detector (YOLOv12) to generate box prompts for TinySAM on salient foreground objects, and supplements uncovered regions with sparse point prompts sampled only where YOLO-guided masks provide no coverage. On COCO val2017, the hybrid system substantially improves class-agnostic coverage (AR from 16.4% to 77.1%, mIoU from 19.2% to 67.8%) while reducing end-to-end runtime from 49.20s/image to 10.39s/image (4.7x) on an Apple M1 Pro CPU. These results suggest detector-guided prompting combined with targeted sparse sampling as an effective alternative to dense "segment-everything" prompting for practical full-scene segmentation.