Compositional Model based Fisher Vector Coding for Image Classification

TL;DR

This paper introduces a compositional approach to Fisher vector coding for image classification, replacing the traditional GMM with a model that combines multiple components for better feature representation.

Contribution

It proposes a novel compositional generative model for Fisher vector coding, enhancing the expressive power over traditional GMM-based methods.

Findings

Enhanced feature representation capacity

Improved image classification accuracy

More flexible generative modeling

Abstract

Deriving from the gradient vector of a generative model of local features, Fisher vector coding (FVC) has been identified as an effective coding method for image classification. Most, if not all, FVC implementations employ the Gaussian mixture model (GMM) to depict the generation process of local features. However, the representative power of the GMM could be limited because it essentially assumes that local features can be characterized by a fixed number of feature prototypes and the number of prototypes is usually small in FVC. To handle this limitation, in this paper we break the convention which assumes that a local feature is drawn from one of few Gaussian distributions. Instead, we adopt a compositional mechanism which assumes that a local feature is drawn from a Gaussian distribution whose mean vector is composed as the linear combination of multiple key components and the combination weight is a latent random variable. In this way, we can greatly enhance the representative power of the generative model of FVC. To implement our idea, we designed two particular generative models with such a compositional mechanism.