When VLAD met Hilbert

TL;DR

This paper introduces a kernelized VLAD that handles non-vector descriptors and manifold data, enabling more flexible and powerful image/video representations with improved classification performance.

Contribution

We propose a kernelized VLAD framework that allows aggregation of non-vector descriptors and manifold data, extending VLAD's applicability and effectiveness.

Findings

Kernel VLAD improves classification on manifold-valued data.

Approximate formulations accelerate the VLAD coding process.

Nonlinear VLAD descriptors outperform linear ones on benchmarks.

Abstract

Vectors of Locally Aggregated Descriptors (VLAD) have emerged as powerful image/video representations that compete with or even outperform state-of-the-art approaches on many challenging visual recognition tasks. In this paper, we address two fundamental limitations of VLAD: its requirement for the local descriptors to have vector form and its restriction to linear classifiers due to its high-dimensionality. To this end, we introduce a kernelized version of VLAD. This not only lets us inherently exploit more sophisticated classification schemes, but also enables us to efficiently aggregate non-vector descriptors (e.g., tensors) in the VLAD framework. Furthermore, we propose three approximate formulations that allow us to accelerate the coding process while still benefiting from the properties of kernel VLAD. Our experiments demonstrate the effectiveness of our approach at handling manifold-valued data, such as covariance descriptors, on several classification tasks. Our results also evidence the benefits of our nonlinear VLAD descriptors against the linear ones in Euclidean space using several standard benchmark datasets.