Main Product Detection with Graph Networks for Fashion

TL;DR

This paper introduces a graph network-based model for main product detection in fashion images, leveraging relationships between image regions to improve accuracy, especially when textual data is unavailable.

Contribution

The paper proposes a novel Graph Convolutional Network approach that models relationships between detected regions, outperforming existing methods in various scenarios.

Findings

Outperforms state-of-the-art in main product detection

Effective even without title-input at inference

Shows strong generalization across datasets

Abstract

Computer vision has established a foothold in the online fashion retail industry. Main product detection is a crucial step of vision-based fashion product feed parsing pipelines, focused in identifying the bounding boxes that contain the product being sold in the gallery of images of the product page. The current state-of-the-art approach does not leverage the relations between regions in the image, and treats images of the same product independently, therefore not fully exploiting visual and product contextual information. In this paper we propose a model that incorporates Graph Convolutional Networks (GCN) that jointly represent all detected bounding boxes in the gallery as nodes. We show that the proposed method is better than the state-of-the-art, especially, when we consider the scenario where title-input is missing at inference time and for cross-dataset evaluation, our method outperforms previous approaches by a large margin.