Do Not Return Similarity: Face Recovery with Distance

TL;DR

This paper demonstrates that leaking distance information in face verification systems can lead to exact recovery of user embeddings and photos, posing severe privacy risks.

Contribution

It reveals new privacy vulnerabilities in face verification systems by showing how distance leaks can be exploited to recover user images using a GAN-like model.

Findings

93.65% success rate in recovering face embeddings

Exact recovery of user photos from leaked embeddings

Identification of channels leaking embedding information

Abstract

Machine Learning (ML) already has been integrated into all kinds of systems, helping developers to solve problems with even higher accuracy than human beings. However, when integrating ML models into a system, developers may accidentally take not enough care of the outputs of ML models, mainly because of their unfamiliarity with ML and AI, resulting in severe consequences like hurting data owners' privacy. In this work, we focus on understanding the risks of abusing embeddings of ML models, an important and popular way of using ML. To show the consequence, we reveal several kinds of channels in which embeddings are accidentally leaked. As our study shows, a face verification system deployed by a government organization leaking only distance to authentic users allows an attacker to exactly recover the embedding of the verifier's pre-installed photo. Further, as we discovered, with the leaked embedding, attackers can easily recover the input photo with negligible quality losses, indicating devastating consequences to users' privacy. This is achieved with our devised GAN-like structure model, which showed 93.65% success rate on popular face embedding model under black box assumption.