Hierarchical Conditional Semi-Paired Image-to-Image Translation For Multi-Task Image Defect Correction On Shopping Websites

TL;DR

This paper introduces a hierarchical, semi-paired image-to-image translation model that effectively corrects multiple product image defects across various types on shopping websites, improving image quality and user experience.

Contribution

The proposed unified model leverages an attention mechanism for hierarchical defect correction and combines paired and unpaired data training, addressing scalability and data quality challenges.

Findings

Reduces FID by 24.6% on eight datasets compared to MoNCE.

Achieves 63.2% FID reduction on a shopping website dataset over WS-I2I.

Effectively corrects multiple defect types across diverse product categories.

Abstract

On shopping websites, product images of low quality negatively affect customer experience. Although there are plenty of work in detecting images with different defects, few efforts have been dedicated to correct those defects at scale. A major challenge is that there are thousands of product types and each has specific defects, therefore building defect specific models is unscalable. In this paper, we propose a unified Image-to-Image (I2I) translation model to correct multiple defects across different product types. Our model leverages an attention mechanism to hierarchically incorporate high-level defect groups and specific defect types to guide the network to focus on defect-related image regions. Evaluated on eight public datasets, our model reduces the Frechet Inception Distance (FID) by 24.6% in average compared with MoNCE, the state-of-the-art I2I method. Unlike public data, another practical challenge on shopping websites is that some paired images are of low quality. Therefore we design our model to be semi-paired by combining the L1 loss of paired data with the cycle loss of unpaired data. Tested on a shopping website dataset to correct three image defects, our model reduces (FID) by 63.2% in average compared with WS-I2I, the state-of-the art semi-paired I2I method.