ShadowMaskFormer: Mask Augmented Patch Embeddings for Shadow Removal

TL;DR

ShadowMaskFormer introduces a novel mask-augmented patch embedding for transformer-based shadow removal, effectively emphasizing shadow regions with fewer parameters, and demonstrates superior performance on standard benchmarks.

Contribution

It proposes a simple, effective mask-augmented patch embedding tailored for shadow removal within transformer models, reducing complexity and resource requirements.

Findings

Outperforms state-of-the-art methods on ISTD, ISTD+, and SRD datasets.

Uses fewer model parameters while maintaining high performance.

Effective integration of shadow information at early processing stages.

Abstract

Transformer recently emerged as the de facto model for computer vision tasks and has also been successfully applied to shadow removal. However, these existing methods heavily rely on intricate modifications to the attention mechanisms within the transformer blocks while using a generic patch embedding. As a result, it often leads to complex architectural designs requiring additional computation resources. In this work, we aim to explore the efficacy of incorporating shadow information within the early processing stage. Accordingly, we propose a transformer-based framework with a novel patch embedding that is tailored for shadow removal, dubbed ShadowMaskFormer. Specifically, we present a simple and effective mask-augmented patch embedding to integrate shadow information and promote the model's emphasis on acquiring knowledge for shadow regions. Extensive experiments conducted on the ISTD, ISTD+, and SRD benchmark datasets demonstrate the efficacy of our method against state-of-the-art approaches while using fewer model parameters.g fewer model parameters. Our implementation is available at https://github.com/lizhh268/ShadowMaskFormer.