Circulant Binary Embedding

TL;DR

This paper introduces Circulant Binary Embedding (CBE), a fast and memory-efficient method for high-dimensional binary data encoding using circulant matrices and Fourier transforms, outperforming existing methods.

Contribution

The paper proposes a novel circulant matrix-based binary embedding method with a Fourier-based optimization, significantly reducing computation and storage costs while improving performance.

Findings

CBE reduces time complexity from O(d^2) to O(d log d).

CBE decreases space complexity from O(d^2) to O(d).

CBE outperforms state-of-the-art methods in experiments.

Abstract

Binary embedding of high-dimensional data requires long codes to preserve the discriminative power of the input space. Traditional binary coding methods often suffer from very high computation and storage costs in such a scenario. To address this problem, we propose Circulant Binary Embedding (CBE) which generates binary codes by projecting the data with a circulant matrix. The circulant structure enables the use of Fast Fourier Transformation to speed up the computation. Compared to methods that use unstructured matrices, the proposed method improves the time complexity from $\mathcal{O}(d^2)$ to $\mathcal{O}(d\log{d})$, and the space complexity from $\mathcal{O}(d^2)$ to $\mathcal{O}(d)$ where $d$ is the input dimensionality. We also propose a novel time-frequency alternating optimization to learn data-dependent circulant projections, which alternatively minimizes the objective in original and Fourier domains. We show by extensive experiments that the proposed approach gives much better performance than the state-of-the-art approaches for fixed time, and provides much faster computation with no performance degradation for fixed number of bits.