Relatable Clothing: Detecting Visual Relationships between People and Clothing

TL;DR

This paper introduces a new dataset and a soft attention-based neural network model for classifying whether clothing in images is worn or unworn, achieving high accuracy and generalizability.

Contribution

The paper releases the Relatable Clothing Dataset and proposes a novel soft attention unit for improved worn/unworn clothing classification.

Findings

Achieved 98.55% accuracy on the dataset.

Demonstrated high generalization to unseen clothing articles.

Provided a new benchmark for clothing relationship detection.

Abstract

Detecting visual relationships between people and clothing in an image has been a relatively unexplored problem in the field of computer vision and biometrics. The lack readily available public dataset for ``worn'' and ``unworn'' classification has slowed the development of solutions for this problem. We present the release of the Relatable Clothing Dataset which contains 35287 person-clothing pairs and segmentation masks for the development of ``worn'' and ``unworn'' classification models. Additionally, we propose a novel soft attention unit for performing ``worn'' and ``unworn'' classification using deep neural networks. The proposed soft attention models have an accuracy of upward $98.55\% \pm 0.35\%$ on the Relatable Clothing Dataset and demonstrate high generalizable, allowing us to classify unseen articles of clothing such as high visibility vests as ``worn'' or ``unworn''.