Learnable Cost Volume Using the Cayley Representation

TL;DR

This paper introduces a learnable cost volume using the Cayley representation, enhancing optical flow models by capturing channel correlations and improving accuracy and robustness.

Contribution

It proposes a novel learnable cost volume with a Cayley-based parameterization, allowing better representation and integration into existing models.

Findings

Improves accuracy of state-of-the-art optical flow models.

Enhances robustness against illumination changes, noise, and adversarial attacks.

Lightweight module easily integrated into existing architectures.

Abstract

Cost volume is an essential component of recent deep models for optical flow estimation and is usually constructed by calculating the inner product between two feature vectors. However, the standard inner product in the commonly-used cost volume may limit the representation capacity of flow models because it neglects the correlation among different channel dimensions and weighs each dimension equally. To address this issue, we propose a learnable cost volume (LCV) using an elliptical inner product, which generalizes the standard inner product by a positive definite kernel matrix. To guarantee its positive definiteness, we perform spectral decomposition on the kernel matrix and re-parameterize it via the Cayley representation. The proposed LCV is a lightweight module and can be easily plugged into existing models to replace the vanilla cost volume. Experimental results show that the LCV module not only improves the accuracy of state-of-the-art models on standard benchmarks, but also promotes their robustness against illumination change, noises, and adversarial perturbations of the input signals.