Quality Aware Network for Set to Set Recognition

TL;DR

This paper introduces a quality aware network (QAN) for set-to-set recognition tasks like face verification and re-identification, which automatically learns sample quality to improve recognition accuracy.

Contribution

The paper proposes a novel end-to-end trainable network that learns sample quality scores without explicit labels, enhancing set-to-set recognition performance.

Findings

QAN outperforms existing methods on face verification datasets.

The learned quality scores improve the robustness of set representations.

End-to-end training simplifies the recognition pipeline.

Abstract

This paper targets on the problem of set to set recognition, which learns the metric between two image sets. Images in each set belong to the same identity. Since images in a set can be complementary, they hopefully lead to higher accuracy in practical applications. However, the quality of each sample cannot be guaranteed, and samples with poor quality will hurt the metric. In this paper, the quality aware network (QAN) is proposed to confront this problem, where the quality of each sample can be automatically learned although such information is not explicitly provided in the training stage. The network has two branches, where the first branch extracts appearance feature embedding for each sample and the other branch predicts quality score for each sample. Features and quality scores of all samples in a set are then aggregated to generate the final feature embedding. We show that the two branches can be trained in an end-to-end manner given only the set-level identity annotation. Analysis on gradient spread of this mechanism indicates that the quality learned by the network is beneficial to set-to-set recognition and simplifies the distribution that the network needs to fit. Experiments on both face verification and person re-identification show advantages of the proposed QAN. The source code and network structure can be downloaded at https://github.com/sciencefans/Quality-Aware-Network.