Soft-Hard Attention U-Net Model and Benchmark Dataset for Multiscale Image Shadow Removal

TL;DR

This paper introduces a novel deep learning model called Soft-Hard Attention U-Net (SHAU) for multiscale shadow removal and provides a new complex dataset (MSRD) for benchmarking, significantly improving shadow removal performance.

Contribution

It proposes a new multiscale shadow removal architecture with attention modules and introduces a comprehensive synthetic dataset for better benchmarking.

Findings

SHAU outperforms state-of-the-art methods in shadow removal metrics.

The model improves PSNR by 25.1% and RMSE by 61.3% over existing methods.

The MSRD dataset captures complex shadow patterns for robust evaluation.

Abstract

Effective shadow removal is pivotal in enhancing the visual quality of images in various applications, ranging from computer vision to digital photography. During the last decades physics and machine learning -based methodologies have been proposed; however, most of them have limited capacity in capturing complex shadow patterns due to restrictive model assumptions, neglecting the fact that shadows usually appear at different scales. Also, current datasets used for benchmarking shadow removal are composed of a limited number of images with simple scenes containing mainly uniform shadows cast by single objects, whereas only a few of them include both manual shadow annotations and paired shadow-free images. Aiming to address all these limitations in the context of natural scene imaging, including urban environments with complex scenes, the contribution of this study is twofold: a) it proposes a novel deep learning architecture, named Soft-Hard Attention U-net (SHAU), focusing on multiscale shadow removal; b) it provides a novel synthetic dataset, named Multiscale Shadow Removal Dataset (MSRD), containing complex shadow patterns of multiple scales, aiming to serve as a privacy-preserving dataset for a more comprehensive benchmarking of future shadow removal methodologies. Key architectural components of SHAU are the soft and hard attention modules, which along with multiscale feature extraction blocks enable effective shadow removal of different scales and intensities. The results demonstrate the effectiveness of SHAU over the relevant state-of-the-art shadow removal methods across various benchmark datasets, improving the Peak Signal-to-Noise Ratio and Root Mean Square Error for the shadow area by 25.1% and 61.3%, respectively.