Identifying Ambiguous Similarity Conditions via Semantic Matching

TL;DR

This paper introduces DiscoverNet, a novel method for weakly supervised conditional similarity learning that effectively captures latent semantic conditions in images, with a new evaluation criterion and state-of-the-art results on benchmark datasets.

Contribution

The paper proposes DiscoverNet, a new network architecture with a novel evaluation criterion for better semantic coverage in weakly supervised conditional similarity learning.

Findings

DiscoverNet achieves state-of-the-art performance on UT-Zappos-50k.

It effectively models latent semantic conditions in images.

The new evaluation criterion measures the coverage of semantic conditions.

Abstract

Rich semantics inside an image result in its ambiguous relationship with others, i.e., two images could be similar in one condition but dissimilar in another. Given triplets like "aircraft" is similar to "bird" than "train", Weakly Supervised Conditional Similarity Learning (WS-CSL) learns multiple embeddings to match semantic conditions without explicit condition labels such as "can fly". However, similarity relationships in a triplet are uncertain except providing a condition. For example, the previous comparison becomes invalid once the conditional label changes to "is vehicle". To this end, we introduce a novel evaluation criterion by predicting the comparison's correctness after assigning the learned embeddings to their optimal conditions, which measures how much WS-CSL could cover latent semantics as the supervised model. Furthermore, we propose the Distance Induced Semantic COndition VERification Network (DiscoverNet), which characterizes the instance-instance and triplets-condition relations in a "decompose-and-fuse" manner. To make the learned embeddings cover all semantics, DiscoverNet utilizes a set module or an additional regularizer over the correspondence between a triplet and a condition. DiscoverNet achieves state-of-the-art performance on benchmarks like UT-Zappos-50k and Celeb-A w.r.t. different criteria.