TL;DR
PointRend introduces a novel rendering-inspired neural network module for high-quality image segmentation, achieving crisp boundaries and significant performance improvements on COCO and Cityscapes datasets.
Contribution
It presents a flexible, point-based segmentation method that enhances boundary quality and efficiency by adaptively focusing on challenging regions, building on existing models.
Findings
Achieves state-of-the-art results on COCO and Cityscapes
Produces crisper object boundaries than previous methods
Enables high-resolution outputs efficiently
Abstract
We present a new method for efficient high-quality image segmentation of objects and scenes. By analogizing classical computer graphics methods for efficient rendering with over- and undersampling challenges faced in pixel labeling tasks, we develop a unique perspective of image segmentation as a rendering problem. From this vantage, we present the PointRend (Point-based Rendering) neural network module: a module that performs point-based segmentation predictions at adaptively selected locations based on an iterative subdivision algorithm. PointRend can be flexibly applied to both instance and semantic segmentation tasks by building on top of existing state-of-the-art models. While many concrete implementations of the general idea are possible, we show that a simple design already achieves excellent results. Qualitatively, PointRend outputs crisp object boundaries in regions that are…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
PointRend: Image Segmentation As Rendering· youtube
Taxonomy
MethodsRegion Proposal Network · Dense Connections · Feedforward Network · Group Normalization · Panoptic FPN · Spatial Pyramid Pooling · Random Scaling · Random Horizontal Flip · Linear Warmup · SGD with Momentum
