Fashion Landmark Detection in the Wild

TL;DR

This paper introduces a new fashion landmark detection method using a large dataset and a multi-stage CNN approach, improving fashion image understanding and outperforming previous representations like bounding boxes and joints.

Contribution

The paper presents a novel fashion landmark dataset with 120K images and a cascaded CNN framework for accurate fashion landmark detection.

Findings

The proposed method achieves high accuracy in fashion landmark detection.

Fashion landmarks outperform bounding boxes and joints in attribute prediction.

The approach generalizes well to pose estimation tasks.

Abstract

Visual fashion analysis has attracted many attentions in the recent years. Previous work represented clothing regions by either bounding boxes or human joints. This work presents fashion landmark detection or fashion alignment, which is to predict the positions of functional key points defined on the fashion items, such as the corners of neckline, hemline, and cuff. To encourage future studies, we introduce a fashion landmark dataset with over 120K images, where each image is labeled with eight landmarks. With this dataset, we study fashion alignment by cascading multiple convolutional neural networks in three stages. These stages gradually improve the accuracies of landmark predictions. Extensive experiments demonstrate the effectiveness of the proposed method, as well as its generalization ability to pose estimation. Fashion landmark is also compared to clothing bounding boxes and human joints in two applications, fashion attribute prediction and clothes retrieval, showing that fashion landmark is a more discriminative representation to understand fashion images.