Multi-modal image retrieval with random walk on multi-layer graphs

TL;DR

This paper introduces a novel multi-layer graph-based image retrieval method that integrates multimodal features through random walks, significantly improving image similarity ranking in large datasets.

Contribution

It proposes a new multi-layer graph model with optimized edge weights for effective multimodal image retrieval, outperforming existing methods.

Findings

High retrieval accuracy in real-world datasets

Superior performance over state-of-the-art solutions

Effective multimodal feature integration

Abstract

The analysis of large collections of image data is still a challenging problem due to the difficulty of capturing the true concepts in visual data. The similarity between images could be computed using different and possibly multimodal features such as color or edge information or even text labels. This motivates the design of image analysis solutions that are able to effectively integrate the multi-view information provided by different feature sets. We therefore propose a new image retrieval solution that is able to sort images through a random walk on a multi-layer graph, where each layer corresponds to a different type of information about the image data. We study in depth the design of the image graph and propose in particular an effective method to select the edge weights for the multi-layer graph, such that the image ranking scores are optimised. We then provide extensive experiments in different real-world photo collections, which confirm the high performance of our new image retrieval algorithm that generally surpasses state-of-the-art solutions due to a more meaningful image similarity computation.