ELMformer: Efficient Raw Image Restoration with a Locally Multiplicative Transformer
Jiaqi Ma, Shengyuan Yan, Lefei Zhang, Guoli Wang, Qian Zhang

TL;DR
ELMformer is a novel transformer-based model designed for efficient raw image restoration, utilizing a bi-directional fusion module and locally multiplicative self-attention to outperform existing methods in quality and computational efficiency.
Contribution
The paper introduces ELMformer, a new raw image restoration model with two innovative modules that enhance performance and reduce computational costs compared to prior approaches.
Findings
Achieves highest performance and lowest FLOPs on raw denoising and deblurring benchmarks.
Outperforms ISP-based methods on SIDD benchmark without needing additional sRGB images.
Demonstrates superior generalization and efficiency in raw image restoration tasks.
Abstract
In order to get raw images of high quality for downstream Image Signal Process (ISP), in this paper we present an Efficient Locally Multiplicative Transformer called ELMformer for raw image restoration. ELMformer contains two core designs especially for raw images whose primitive attribute is single-channel. The first design is a Bi-directional Fusion Projection (BFP) module, where we consider both the color characteristics of raw images and spatial structure of single-channel. The second one is that we propose a Locally Multiplicative Self-Attention (L-MSA) scheme to effectively deliver information from the local space to relevant parts. ELMformer can efficiently reduce the computational consumption and perform well on raw image restoration tasks. Enhanced by these two core designs, ELMformer achieves the highest performance and keeps the lowest FLOPs on raw denoising and raw…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsMulti-Head Attention · Attention Is All You Need · Linear Layer · Absolute Position Encodings · Layer Normalization · Softmax · Adam · Position-Wise Feed-Forward Layer · Dropout · Dense Connections
