Inpaint Anything: Segment Anything Meets Image Inpainting

TL;DR

Inpaint Anything (IA) introduces a user-friendly, mask-free image inpainting paradigm that combines segmentation and generative models to enable object removal, filling with text prompts, and replacement within images.

Contribution

The paper presents the first mask-free inpainting framework based on Segment-Anything Model, enabling intuitive editing through clicking and filling, with versatile object removal, filling, and replacement features.

Findings

Supports object removal with smoothing

Enables text-guided image filling

Allows object replacement with generated content

Abstract

Modern image inpainting systems, despite the significant progress, often struggle with mask selection and holes filling. Based on Segment-Anything Model (SAM), we make the first attempt to the mask-free image inpainting and propose a new paradigm of ``clicking and filling'', which is named as Inpaint Anything (IA). The core idea behind IA is to combine the strengths of different models in order to build a very powerful and user-friendly pipeline for solving inpainting-related problems. IA supports three main features: (i) Remove Anything: users could click on an object and IA will remove it and smooth the ``hole'' with the context; (ii) Fill Anything: after certain objects removal, users could provide text-based prompts to IA, and then it will fill the hole with the corresponding generative content via driving AIGC models like Stable Diffusion; (iii) Replace Anything: with IA, users have another option to retain the click-selected object and replace the remaining background with the newly generated scenes. We are also very willing to help everyone share and promote new projects based on our Inpaint Anything (IA). Our codes are available at https://github.com/geekyutao/Inpaint-Anything.