Interactive Image Manipulation with Complex Text Instructions
Ryugo Morita, Zhiqiang Zhang, Man M. Ho, Jinjia Zhou

TL;DR
This paper introduces an interactive image manipulation method guided by complex text instructions, allowing precise, flexible edits including object resizing, removal, and background replacement, with real-time performance.
Contribution
The proposed approach uniquely separates text-relevant and irrelevant content, employs super-resolution for region enlargement, and provides an interactive segmentation interface, advancing the accuracy and complexity of text-guided image editing.
Findings
Outperforms state-of-the-art methods in accuracy and flexibility.
Enables complex manipulations like resizing, removal, and background change.
Operates in real-time with extensive experimental validation.
Abstract
Recently, text-guided image manipulation has received increasing attention in the research field of multimedia processing and computer vision due to its high flexibility and controllability. Its goal is to semantically manipulate parts of an input reference image according to the text descriptions. However, most of the existing works have the following problems: (1) text-irrelevant content cannot always be maintained but randomly changed, (2) the performance of image manipulation still needs to be further improved, (3) only can manipulate descriptive attributes. To solve these problems, we propose a novel image manipulation method that interactively edits an image using complex text instructions. It allows users to not only improve the accuracy of image manipulation but also achieve complex tasks such as enlarging, dwindling, or removing objects and replacing the background with the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Interactive Image Manipulation with Complex Text Instructions· youtube
Taxonomy
TopicsAdvanced Image Processing Techniques · Image Processing Techniques and Applications · Digital Media Forensic Detection
