MEFLUT: Unsupervised 1D Lookup Tables for Multi-exposure Image Fusion

TL;DR

This paper presents MEFLUT, an unsupervised method for multi-exposure image fusion using learned 1D lookup tables with attention mechanisms, achieving high quality and efficiency, and outperforming state-of-the-art methods.

Contribution

Introduction of a novel unsupervised approach using 1D lookup tables and attention mechanisms for high-quality multi-exposure image fusion.

Findings

Outperforms state-of-the-art in quality and efficiency.

Runs in less than 4ms for 4K images on GPU.

Successfully deployed in millions of Android devices.

Abstract

In this paper, we introduce a new approach for high-quality multi-exposure image fusion (MEF). We show that the fusion weights of an exposure can be encoded into a 1D lookup table (LUT), which takes pixel intensity value as input and produces fusion weight as output. We learn one 1D LUT for each exposure, then all the pixels from different exposures can query 1D LUT of that exposure independently for high-quality and efficient fusion. Specifically, to learn these 1D LUTs, we involve attention mechanism in various dimensions including frame, channel and spatial ones into the MEF task so as to bring us significant quality improvement over the state-of-the-art (SOTA). In addition, we collect a new MEF dataset consisting of 960 samples, 155 of which are manually tuned by professionals as ground-truth for evaluation. Our network is trained by this dataset in an unsupervised manner. Extensive experiments are conducted to demonstrate the effectiveness of all the newly proposed components, and results show that our approach outperforms the SOTA in our and another representative dataset SICE, both qualitatively and quantitatively. Moreover, our 1D LUT approach takes less than 4ms to run a 4K image on a PC GPU. Given its high quality, efficiency and robustness, our method has been shipped into millions of Android mobiles across multiple brands world-wide. Code is available at: https://github.com/Hedlen/MEFLUT.