Shadow Removal by High-Quality Shadow Synthesis

TL;DR

This paper introduces HQSS, a novel framework for high-quality shadow image synthesis that improves shadow removal by generating more authentic and detailed pseudo shadow images, outperforming existing methods.

Contribution

The paper proposes a new shadow synthesis framework, HQSS, which decouples images into shadow and non-shadow regions and employs specialized losses to produce more realistic shadow images.

Findings

HQSS outperforms state-of-the-art methods on multiple datasets.

The framework effectively retains shadow characteristics and image details.

HQSS improves the quality of pseudo shadow images for shadow removal tasks.

Abstract

Most shadow removal methods rely on the invasion of training images associated with laborious and lavish shadow region annotations, leading to the increasing popularity of shadow image synthesis. However, the poor performance also stems from these synthesized images since they are often shadow-inauthentic and details-impaired. In this paper, we present a novel generation framework, referred to as HQSS, for high-quality pseudo shadow image synthesis. The given image is first decoupled into a shadow region identity and a non-shadow region identity. HQSS employs a shadow feature encoder and a generator to synthesize pseudo images. Specifically, the encoder extracts the shadow feature of a region identity which is then paired with another region identity to serve as the generator input to synthesize a pseudo image. The pseudo image is expected to have the shadow feature as its input shadow feature and as well as a real-like image detail as its input region identity. To fulfill this goal, we design three learning objectives. When the shadow feature and input region identity are from the same region identity, we propose a self-reconstruction loss that guides the generator to reconstruct an identical pseudo image as its input. When the shadow feature and input region identity are from different identities, we introduce an inter-reconstruction loss and a cycle-reconstruction loss to make sure that shadow characteristics and detail information can be well retained in the synthesized images. Our HQSS is observed to outperform the state-of-the-art methods on ISTD dataset, Video Shadow Removal dataset, and SRD dataset. The code is available at https://github.com/zysxmu/HQSS.