Multi-Granularity Representation Learning for Sketch-based Dynamic Face Image Retrieval

TL;DR

This paper introduces a multigranularity representation learning approach for sketchless face image retrieval, improving early-stage retrieval performance by capturing different region granularities of partial sketches and images.

Contribution

The study proposes a novel multigranularity learning method that enhances sketchless face retrieval by effectively representing partial sketches at multiple levels of detail.

Findings

Outperforms state-of-the-art methods in early retrieval scenarios

Demonstrates effectiveness on two accessible datasets

Provides a new framework for partial sketch and face image matching

Abstract

In specific scenarios, face sketch can be used to identify a person. However, drawing a face sketch often requires exceptional skill and is time-consuming, limiting its widespread applications in actual scenarios. The new framework of sketch less face image retrieval (SLFIR)[1] attempts to overcome the barriers by providing a means for humans and machines to interact during the drawing process. Considering SLFIR problem, there is a large gap between a partial sketch with few strokes and any whole face photo, resulting in poor performance at the early stages. In this study, we propose a multigranularity (MG) representation learning (MGRL) method to address the SLFIR problem, in which we learn the representation of different granularity regions for a partial sketch, and then, by combining all MG regions of the sketches and images, the final distance was determined. In the experiments, our method outperformed state-of-the-art baselines in terms of early retrieval on two accessible datasets. Codes are available at https://github.com/ddw2AIGROUP2CQUPT/MGRL.