Compact Approximation for Polynomial of Covariance Feature

TL;DR

This paper introduces a compact polynomial approximation method for covariance features in CNNs, enabling efficient, end-to-end trainable pooling with comparable accuracy and fewer dimensions on fine-grained image recognition tasks.

Contribution

It extends compact bilinear pooling to polynomial covariance features, including a novel approximation for the matrix square root, improving efficiency and stability.

Findings

Achieves comparable accuracy with fewer feature dimensions.

Demonstrates effectiveness on fine-grained image recognition datasets.

Provides a faster, more stable training method for covariance pooling.

Abstract

Covariance pooling is a feature pooling method with good classification accuracy. Because covariance features consist of second-order statistics, the scale of the feature elements are varied. Therefore, normalizing covariance features using a matrix square root affects the performance improvement. When pooling methods are applied to local features extracted from CNN models, the accuracy increases when the pooling function is back-propagatable and the feature-extraction model is learned in an end-to-end manner. Recently, the iterative polynomial approximation method for the matrix square root of a covariance feature was proposed, and resulted in a faster and more stable training than the methods based on singular-value decomposition. In this paper, we propose an extension of compact bilinear pooling, which is a compact approximation of the standard covariance feature, to the polynomials of the covariance feature. Subsequently, we apply the proposed approximation to the polynomial corresponding to the matrix square root to obtain a compact approximation for the square root of the covariance feature. Our method approximates a higher-dimensional polynomial of a covariance by the weighted sum of the approximate features corresponding to a pair of local features based on the similarity of the local features. We apply our method for standard fine-grained image recognition datasets and demonstrate that the proposed method shows comparable accuracy with fewer dimensions than the original feature.