# Outfit Compatibility Prediction and Diagnosis with Multi-Layered Comparison Network

## TL;DR

This paper introduces a multi-layered comparison network for predicting and diagnosing outfit compatibility, leveraging hierarchical features and pairwise similarities, supported by a new dataset, Polyvore-T, demonstrating improved accuracy and interpretability.

## Contribution

The paper presents a novel end-to-end framework that predicts and diagnoses outfit compatibility using multi-layered feature comparison and gradient-based diagnosis, along with a new dataset Polyvore-T.

## Key findings

- Outperforms prior methods in compatibility prediction accuracy.
- Provides effective diagnosis of incompatible factors.
- Achieves better interpretability through hierarchical feature analysis.

## Abstract

Existing works about fashion outfit compatibility focus on predicting the overall compatibility of a set of fashion items with their information from different modalities. However, there are few works explore how to explain the prediction, which limits the persuasiveness and effectiveness of the model. In this work, we propose an approach to not only predict but also diagnose the outfit compatibility. We introduce an end-to-end framework for this goal, which features for: (1) The overall compatibility is learned from all type-specified pairwise similarities between items, and the backpropagation gradients are used to diagnose the incompatible factors. (2) We leverage the hierarchy of CNN and compare the features at different layers to take into account the compatibilities of different aspects from the low level (such as color, texture) to the high level (such as style). To support the proposed method, we build a new type-specified outfit dataset named Polyvore-T based on Polyvore dataset. We compare our method with the prior state-of-the-art in two tasks: outfit compatibility prediction and fill-in-the-blank. Experiments show that our approach has advantages in both prediction performance and diagnosis ability.