QSAM-Net: Rain streak removal by quaternion neural network with self-attention module

TL;DR

QSAM-Net is a novel quaternion neural network with self-attention that effectively removes rain streaks from images, requiring fewer parameters and enabling real-time performance on edge devices.

Contribution

It introduces a multi-stage multiscale quaternion neural network with self-attention for rain removal, significantly reducing parameters while enhancing image quality.

Findings

Reduces model parameters by nearly 4 times compared to prior methods.

Improves visual quality and object detection accuracy in rainy images.

Enables near real-time processing suitable for edge devices.

Abstract

Images captured in real-world applications in remote sensing, image or video retrieval, and outdoor surveillance suffer degraded quality introduced by poor weather conditions. Conditions such as rain and mist, introduce artifacts that make visual analysis challenging and limit the performance of high-level computer vision methods. For time-critical applications where a rapid response is necessary, it becomes crucial to develop algorithms that automatically remove rain, without diminishing the quality of the image contents. This article aims to develop a novel quaternion multi-stage multiscale neural network with a self-attention module called QSAM-Net to remove rain streaks. The novelty of this algorithm is that it requires significantly fewer parameters by a factor of 3.98, over prior methods, while improving visual quality. This is demonstrated by the extensive evaluation and benchmarking on synthetic and real-world rainy images. This feature of QSAM-Net makes the network suitable for implementation on edge devices and applications requiring near real-time performance. The experiments demonstrate that by improving the visual quality of images. In addition, object detection accuracy and training speed are also improved.