Multi-Task Curriculum Transfer Deep Learning of Clothing Attributes

TL;DR

This paper introduces a Multi-Task Curriculum Transfer deep learning framework that effectively transfers knowledge from controlled shop images to in-the-wild street images for detailed clothing attribute recognition, especially with limited training data.

Contribution

The paper proposes a novel multi-task curriculum transfer method that leverages multiple web annotation sources and staged learning to improve clothing attribute recognition in unconstrained images.

Findings

Outperforms state-of-the-art on X-Domain benchmark

Excels with small training datasets

Demonstrates effective multi-source transfer learning

Abstract

Recognising detailed clothing characteristics (fine-grained attributes) in unconstrained images of people in-the-wild is a challenging task for computer vision, especially when there is only limited training data from the wild whilst most data available for model learning are captured in well-controlled environments using fashion models (well lit, no background clutter, frontal view, high-resolution). In this work, we develop a deep learning framework capable of model transfer learning from well-controlled shop clothing images collected from web retailers to in-the-wild images from the street. Specifically, we formulate a novel Multi-Task Curriculum Transfer (MTCT) deep learning method to explore multiple sources of different types of web annotations with multi-labelled fine-grained attributes. Our multi-task loss function is designed to extract more discriminative representations in training by jointly learning all attributes, and our curriculum strategy exploits the staged easy-to-complex transfer learning motivated by cognitive studies. We demonstrate the advantages of the MTCT model over the state-of-the-art methods on the X-Domain benchmark, a large scale clothing attribute dataset. Moreover, we show that the MTCT model has a notable advantage over contemporary models when the training data size is small.