Visual Word Selection without Re-Coding and Re-Pooling

TL;DR

This paper introduces a novel method for selecting a subset of visual words from a large codebook in Bag-of-Words models, eliminating the need for re-coding and re-pooling, thus reducing computational costs in computer vision tasks.

Contribution

It proposes formulations for hard and soft coding schemes that enable efficient subset selection without re-coding or re-pooling, improving computational efficiency in BoW-based applications.

Findings

Effective subset selection demonstrated on 15 Scenes benchmark.

Reduces computational complexity compared to brute-force methods.

Applicable to both hard and soft coding schemes.

Abstract

The Bag-of-Words (BoW) representation is widely used in computer vision. The size of the codebook impacts the time and space complexity of the applications that use BoW. Thus, given a training set for a particular computer vision task, a key problem is pruning a large codebook to select only a subset of visual words. Evaluating possible selections of words to be included in the pruned codebook can be computationally prohibitive; in a brute-force scheme, evaluating each pruned codebook requires re-coding of all features extracted from training images to words in the candidate codebook and then re-pooling the words to obtain a representation of each image, e.g., histogram of visual word frequencies. In this paper, a method is proposed that selects and evaluates a subset of words from an initially large codebook, without the need for re-coding or re-pooling. Formulations are proposed for two commonly-used schemes: hard and soft (kernel) coding of visual words with average-pooling. The effectiveness of these formulations is evaluated on the 15 Scenes and Caltech 10 benchmarks.