Image Retrieval with Fisher Vectors of Binary Features

TL;DR

This paper introduces a Fisher vector approach for binary features in image retrieval, deriving a closed-form approximation and demonstrating improved accuracy over traditional methods.

Contribution

It develops a novel Fisher vector representation for binary features using Bernoulli mixture models and proposes an acceleration technique for practical computation.

Findings

Fisher vector significantly outperforms bag of binary words in retrieval accuracy.

The closed-form approximation enables efficient computation of Fisher vectors for binary features.

Experiments validate the effectiveness of the proposed method in image retrieval tasks.

Abstract

Recently, the Fisher vector representation of local features has attracted much attention because of its effectiveness in both image classification and image retrieval. Another trend in the area of image retrieval is the use of binary features such as ORB, FREAK, and BRISK. Considering the significant performance improvement for accuracy in both image classification and retrieval by the Fisher vector of continuous feature descriptors, if the Fisher vector were also to be applied to binary features, we would receive similar benefits in binary feature based image retrieval and classification. In this paper, we derive the closed-form approximation of the Fisher vector of binary features modeled by the Bernoulli mixture model. We also propose accelerating the Fisher vector by using the approximate value of posterior probability. Experiments show that the Fisher vector representation significantly improves the accuracy of image retrieval compared with a bag of binary words approach.