Super-Resolving Blurry Images with Events

TL;DR

This paper presents EBSR-Net, a novel event-based super-resolution network that uses high-temporal-resolution events to enhance blurry images, achieving superior results in super-resolving motion-blurred images.

Contribution

The paper introduces a multi-scale event representation, a symmetric cross-modal attention module, and an intermodal residual group with Swin Transformer blocks for improved super-resolution.

Findings

Outperforms state-of-the-art methods in super-resolving blurry images.

Effectively leverages event data to mitigate motion blur.

Achieves high-quality high-resolution image reconstruction.

Abstract

Super-resolution from motion-blurred images poses a significant challenge due to the combined effects of motion blur and low spatial resolution. To address this challenge, this paper introduces an Event-based Blurry Super Resolution Network (EBSR-Net), which leverages the high temporal resolution of events to mitigate motion blur and improve high-resolution image prediction. Specifically, we propose a multi-scale center-surround event representation to fully capture motion and texture information inherent in events. Additionally, we design a symmetric cross-modal attention module to fully exploit the complementarity between blurry images and events. Furthermore, we introduce an intermodal residual group composed of several residual dense Swin Transformer blocks, each incorporating multiple Swin Transformer layers and a residual connection, to extract global context and facilitate inter-block feature aggregation. Extensive experiments show that our method compares favorably against state-of-the-art approaches and achieves remarkable performance.