Differential Geometric Retrieval of Deep Features

TL;DR

This paper explores advanced geometric and information-theoretic metrics for deep feature-based image retrieval, demonstrating that Wasserstein geometry offers superior retrieval performance despite higher computational costs.

Contribution

It introduces a novel combination of deep learning features with Riemannian and Wasserstein geometric metrics for improved image retrieval accuracy.

Findings

Wasserstein-based metrics outperform other divergence measures in retrieval tasks.

Approximate mixture models reduce computational effort for Wasserstein distance calculation.

Affine invariant metrics provide robust comparison of optimal transport-based image features.

Abstract

Comparing images to recommend items from an image-inventory is a subject of continued interest. Added with the scalability of deep-learning architectures the once `manual' job of hand-crafting features have been largely alleviated, and images can be compared according to features generated from a deep convolutional neural network. In this paper, we compare distance metrics (and divergences) to rank features generated from a neural network, for content-based image retrieval. Specifically, after modelling individual images using approximations of mixture models or sparse covariance estimators, we resort to their information-theoretic and Riemann geometric comparisons. We show that using approximations of mixture models enable us to compute a distance measure based on the Wasserstein metric that requires less effort than other computationally intensive optimal transport plans; finally, an affine invariant metric is used to compare the optimal transport metric to its Riemann geometric counterpart -- we conclude that although expensive, retrieval metric based on Wasserstein geometry is more suitable than information theoretic comparison of images. In short, we combine GPU scalability in learning deep feature vectors with statistically efficient metrics that we foresee being utilised in a commercial setting.