De-Hashing: Server-Side Context-Aware Feature Reconstruction for Mobile Visual Search

TL;DR

This paper introduces a server-assisted de-hashing method that reconstructs detailed Bag-of-Words features from binary codes in mobile visual search, leveraging contextual data to improve retrieval accuracy while minimizing data transmission.

Contribution

It presents a novel de-hashing approach that reconstructs BoW features from binary codes using server resources and contextual information, enhancing mobile visual search performance.

Findings

Achieves retrieval accuracy comparable to traditional BoW methods.

Transmits significantly fewer bits from mobile devices.

Utilizes contextual data to improve feature reconstruction.

Abstract

Due to the prevalence of mobile devices, mobile search becomes a more convenient way than desktop search. Different from the traditional desktop search, mobile visual search needs more consideration for the limited resources on mobile devices (e.g., bandwidth, computing power, and memory consumption). The state-of-the-art approaches show that bag-of-words (BoW) model is robust for image and video retrieval; however, the large vocabulary tree might not be able to be loaded on the mobile device. We observe that recent works mainly focus on designing compact feature representations on mobile devices for bandwidth-limited network (e.g., 3G) and directly adopt feature matching on remote servers (cloud). However, the compact (binary) representation might fail to retrieve target objects (images, videos). Based on the hashed binary codes, we propose a de-hashing process that reconstructs BoW by leveraging the computing power of remote servers. To mitigate the information loss from binary codes, we further utilize contextual information (e.g., GPS) to reconstruct a context-aware BoW for better retrieval results. Experiment results show that the proposed method can achieve competitive retrieval accuracy as BoW while only transmitting few bits from mobile devices.