Looking at Outfit to Parse Clothing

TL;DR

This paper enhances fully-convolutional neural networks for clothing parsing by introducing an outfit encoder and CRF, achieving state-of-the-art results and enabling applications like dress-up image retrieval.

Contribution

It proposes a novel outfit encoder side-branch and integrates CRF for improved clothing parsing without extra supervision.

Findings

Achieves state-of-the-art performance on Fashionista and CFPD datasets.

Demonstrates the effectiveness of the outfit encoder in dress-up image retrieval.

Provides a new annotation tool for clothing datasets.

Abstract

This paper extends fully-convolutional neural networks (FCN) for the clothing parsing problem. Clothing parsing requires higher-level knowledge on clothing semantics and contextual cues to disambiguate fine-grained categories. We extend FCN architecture with a side-branch network which we refer outfit encoder to predict a consistent set of clothing labels to encourage combinatorial preference, and with conditional random field (CRF) to explicitly consider coherent label assignment to the given image. The empirical results using Fashionista and CFPD datasets show that our model achieves state-of-the-art performance in clothing parsing, without additional supervision during training. We also study the qualitative influence of annotation on the current clothing parsing benchmarks, with our Web-based tool for multi-scale pixel-wise annotation and manual refinement effort to the Fashionista dataset. Finally, we show that the image representation of the outfit encoder is useful for dress-up image retrieval application.