DualHash: A Stochastic Primal-Dual Algorithm with Theoretical Guarantee for Deep Hashing

TL;DR

DualHash introduces a stochastic primal-dual algorithm with theoretical guarantees for deep hashing, effectively handling discrete regularizations and improving retrieval performance with proven convergence bounds.

Contribution

It proposes a novel primal-dual hashing algorithm that transforms nonconvex regularization into the dual space, enabling closed-form solutions and providing convergence guarantees.

Findings

DualHash achieves superior retrieval accuracy on image databases.

The algorithm has complexity bounds of O(ε^{-4}) and O(ε^{-3}) with variance reduction.

Experiments validate the effectiveness of the proposed method.

Abstract

Deep hashing converts high-dimensional feature vectors into compact binary codes, enabling efficient large-scale retrieval. A fundamental challenge in deep hashing stems from the discrete nature of quantization in generating the codes. W-type regularizations, such as $||z|-1|$, have been proven effective as they encourage variables toward binary values. However, existing methods often directly optimize these regularizations without convergence guarantees. While proximal gradient methods offer a promising solution, the coupling between W-type regularizers and neural network outputs results in composite forms that generally lack closed-form proximal solutions. In this paper, we present a stochastic primal-dual hashing algorithm, referred to as DualHash, that provides rigorous complexity bounds. Using Fenchel duality, we partially transform the nonconvex W-type regularization optimization into the dual space, which results in a proximal operator that admits closed-form solutions. We derive two algorithm instances: a momentum-accelerated version with $\mathcal{O}(\varepsilon^{-4})$ complexity and an improved $\mathcal{O}(\varepsilon^{-3})$ version using variance reduction. Experiments on three image retrieval databases demonstrate the superior performance of DualHash.