AMES: Asymmetric and Memory-Efficient Similarity Estimation for Instance-level Retrieval

TL;DR

This paper introduces AMES, a transformer-based model for instance-level image retrieval that achieves high accuracy with extremely low memory usage by employing asymmetric similarity estimation and adaptable descriptor representation.

Contribution

The work presents a novel memory-efficient similarity estimation model that uses asymmetric descriptors and adapts to varying descriptor counts, outperforming existing methods in both accuracy and memory footprint.

Findings

Outperforms state-of-the-art methods on standard benchmarks.

Achieves high retrieval accuracy with only 1KB memory per image.

Demonstrates effective asymmetric similarity estimation in image retrieval.

Abstract

This work investigates the problem of instance-level image retrieval re-ranking with the constraint of memory efficiency, ultimately aiming to limit memory usage to 1KB per image. Departing from the prevalent focus on performance enhancements, this work prioritizes the crucial trade-off between performance and memory requirements. The proposed model uses a transformer-based architecture designed to estimate image-to-image similarity by capturing interactions within and across images based on their local descriptors. A distinctive property of the model is the capability for asymmetric similarity estimation. Database images are represented with a smaller number of descriptors compared to query images, enabling performance improvements without increasing memory consumption. To ensure adaptability across different applications, a universal model is introduced that adjusts to a varying number of local descriptors during the testing phase. Results on standard benchmarks demonstrate the superiority of our approach over both hand-crafted and learned models. In particular, compared with current state-of-the-art methods that overlook their memory footprint, our approach not only attains superior performance but does so with a significantly reduced memory footprint. The code and pretrained models are publicly available at: https://github.com/pavelsuma/ames