Textual Prompt Guided Image Restoration
Qiuhai Yan, Aiwen Jiang, Kang Chen, Long Peng, Qiaosi Yi, and Chunjie Zhang

TL;DR
This paper introduces a novel image restoration approach guided by textual prompts, leveraging fine-tuned BERT and specialized modules to improve performance and control in denoising, dehazing, and deraining tasks.
Contribution
It proposes a new textual prompt guided model that effectively integrates human instructions into low-level image restoration, enhancing accuracy and controllability without increasing complexity.
Findings
Outperforms state-of-the-art methods on denoising, dehazing, and deraining datasets.
Achieves superior restoration quality with accurate degradation removal.
Provides a natural and precise control mechanism via textual prompts.
Abstract
Image restoration has always been a cutting-edge topic in the academic and industrial fields of computer vision. Since degradation signals are often random and diverse, "all-in-one" models that can do blind image restoration have been concerned in recent years. Early works require training specialized headers and tails to handle each degradation of concern, which are manually cumbersome. Recent works focus on learning visual prompts from data distribution to identify degradation type. However, the prompts employed in most of models are non-text, lacking sufficient emphasis on the importance of human-in-the-loop. In this paper, an effective textual prompt guided image restoration model has been proposed. In this model, task-specific BERT is fine-tuned to accurately understand user's instructions and generating textual prompt guidance. Depth-wise multi-head transposed attentions and gated…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsImage Enhancement Techniques · Image and Signal Denoising Methods · Advanced Image Processing Techniques
MethodsMulti-Head Attention · Attention Is All You Need · 1x1 Convolution · Refunds@Expedia|||How do I get a full refund from Expedia? · Linear Layer · Dense Connections · Focus · Attention Dropout · Linear Warmup With Linear Decay · WordPiece
