The Conditional Cauchy-Schwarz Divergence with Applications to Time-Series Data and Sequential Decision Making

TL;DR

This paper introduces a new conditional Cauchy-Schwarz divergence measure that effectively quantifies distribution closeness, offering advantages over existing methods, and demonstrates its utility in time series clustering and sequential decision making.

Contribution

It extends the classic CS divergence to conditional distributions, providing a simple estimation method and demonstrating superior performance in time series and sequential inference tasks.

Findings

Outperforms conditional KL divergence and MMD in various tasks.

Effective in time series clustering and uncertainty-guided exploration.

Offers lower computational complexity and higher statistical power.

Abstract

The Cauchy-Schwarz (CS) divergence was developed by Pr\'{i}ncipe et al. in 2000. In this paper, we extend the classic CS divergence to quantify the closeness between two conditional distributions and show that the developed conditional CS divergence can be simply estimated by a kernel density estimator from given samples. We illustrate the advantages (e.g., rigorous faithfulness guarantee, lower computational complexity, higher statistical power, and much more flexibility in a wide range of applications) of our conditional CS divergence over previous proposals, such as the conditional KL divergence and the conditional maximum mean discrepancy. We also demonstrate the compelling performance of conditional CS divergence in two machine learning tasks related to time series data and sequential inference, namely time series clustering and uncertainty-guided exploration for sequential decision making. The code of conditional CS divergence is available at https://github.com/SJYuCNEL/conditional_CS_divergence.