HRSeg: High-Resolution Visual Perception and Enhancement for Reasoning Segmentation

TL;DR

HRSeg introduces a high-resolution perception and enhancement framework for reasoning segmentation, significantly improving the accuracy of object segmentation based on implicit instructions by leveraging detailed visual information.

Contribution

The paper proposes HRSeg, a novel high-resolution perception and enhancement model that effectively processes and refines visual features for reasoning segmentation tasks.

Findings

HRSeg outperforms existing methods on multiple benchmarks.

High-resolution modules improve segmentation accuracy.

Efficient high-resolution processing reduces computational costs.

Abstract

The reasoning segmentation task involves segmenting objects within an image by interpreting implicit user instructions, which may encompass subtleties such as contextual cues and open-world knowledge. Despite significant advancements made by existing approaches, they remain constrained by low perceptual resolution, as visual encoders are typically pre-trained at lower resolutions. Furthermore, simply interpolating the positional embeddings of visual encoders to enhance perceptual resolution yields only marginal performance improvements while incurring substantial computational costs. To address this, we propose HRSeg, an efficient model with high-resolution fine-grained perception. It features two key innovations: High-Resolution Perception (HRP) and High-Resolution Enhancement (HRE). The HRP module processes high-resolution images through cropping, integrating local and global features for multi-granularity quality. The HRE module enhances mask features by integrating fine-grained information from high-resolution images, refining their alignment with text features for precise segmentation. Extensive ablation studies validate the effectiveness of our modules, while comprehensive experiments on multiple benchmark datasets demonstrate HRSeg's superior performance.