Rescuing Deep Hashing from Dead Bits Problem

TL;DR

This paper introduces a gradient amplifier and an error-aware quantization loss to address the Dead Bits Problem in deep hashing, improving the optimization of discrete hash bits for better image retrieval accuracy.

Contribution

It proposes novel techniques—gradient amplifier and error-aware quantization loss—that effectively mitigate Dead Bits Problem in deep hashing methods.

Findings

The methods improve retrieval accuracy on three datasets.

The techniques reduce the saturation of hash bits in activation functions.

Experimental results validate the effectiveness of the proposed approaches.

Abstract

Deep hashing methods have shown great retrieval accuracy and efficiency in large-scale image retrieval. How to optimize discrete hash bits is always the focus in deep hashing methods. A common strategy in these methods is to adopt an activation function, e.g. $\operatorname{sigmoid}(\cdot)$ or $\operatorname{tanh}(\cdot)$, and minimize a quantization loss to approximate discrete values. However, this paradigm may make more and more hash bits stuck into the wrong saturated area of the activation functions and never escaped. We call this problem "Dead Bits Problem~(DBP)". Besides, the existing quantization loss will aggravate DBP as well. In this paper, we propose a simple but effective gradient amplifier which acts before activation functions to alleviate DBP. Moreover, we devise an error-aware quantization loss to further alleviate DBP. It avoids the negative effect of quantization loss based on the similarity between two images. The proposed gradient amplifier and error-aware quantization loss are compatible with a variety of deep hashing methods. Experimental results on three datasets demonstrate the efficiency of the proposed gradient amplifier and the error-aware quantization loss.