Referring Image Matting
Jizhizi Li, Jing Zhang, Dacheng Tao

TL;DR
This paper introduces Referring Image Matting (RIM), a new task that extracts specific object mattes based on natural language descriptions, supported by a large dataset and a novel baseline method.
Contribution
The paper presents a new RIM task, a large-scale dataset RefMatte, and a baseline method CLIPMat, advancing image matting with language-guided object extraction.
Findings
CLIPMat outperforms existing methods on RefMatte.
RefMatte dataset contains 230 categories and 47,500 images.
CLIPMat effectively handles complex natural language descriptions.
Abstract
Different from conventional image matting, which either requires user-defined scribbles/trimap to extract a specific foreground object or directly extracts all the foreground objects in the image indiscriminately, we introduce a new task named Referring Image Matting (RIM) in this paper, which aims to extract the meticulous alpha matte of the specific object that best matches the given natural language description, thus enabling a more natural and simpler instruction for image matting. First, we establish a large-scale challenging dataset RefMatte by designing a comprehensive image composition and expression generation engine to automatically produce high-quality images along with diverse text attributes based on public datasets. RefMatte consists of 230 object categories, 47,500 images, 118,749 expression-region entities, and 474,996 expressions. Additionally, we construct a real-world…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Generative Adversarial Networks and Image Synthesis · Image Enhancement Techniques
MethodsTest
