Ultra-High-Definition Dynamic Multi-Exposure Image Fusion via Infinite Pixel Learning

TL;DR

This paper introduces Infinite Pixel Learning, a novel method for real-time UHD multi-exposure image fusion in dynamic scenes on consumer GPUs, inspired by LLMs, using sequence slicing, attention cache, and compression.

Contribution

It proposes a new learning paradigm for UHD dynamic scene image fusion that overcomes sequence length limitations with innovative attention caching and compression techniques.

Findings

Achieves real-time fusion (>40fps) on a single GPU.

Maintains high visual quality in UHD multi-exposure images.

Provides a new benchmark for UHD dynamic scene fusion.

Abstract

With the continuous improvement of device imaging resolution, the popularity of Ultra-High-Definition (UHD) images is increasing. Unfortunately, existing methods for fusing multi-exposure images in dynamic scenes are designed for low-resolution images, which makes them inefficient for generating high-quality UHD images on a resource-constrained device. To alleviate the limitations of extremely long-sequence inputs, inspired by the Large Language Model (LLM) for processing infinitely long texts, we propose a novel learning paradigm to achieve UHD multi-exposure dynamic scene image fusion on a single consumer-grade GPU, named Infinite Pixel Learning (IPL). The design of our approach comes from three key components: The first step is to slice the input sequences to relieve the pressure generated by the model processing the data stream; Second, we develop an attention cache technique, which is similar to KV cache for infinite data stream processing; Finally, we design a method for attention cache compression to alleviate the storage burden of the cache on the device. In addition, we provide a new UHD benchmark to evaluate the effectiveness of our method. Extensive experimental results show that our method maintains high-quality visual performance while fusing UHD dynamic multi-exposure images in real-time (>40fps) on a single consumer-grade GPU.