Quantum-Inspired Fidelity-based Divergence

TL;DR

This paper introduces a Quantum-Inspired Fidelity-based Divergence (QIF) that offers a more stable and theoretically sound measure of distribution dissimilarity, along with a regularization technique called QR-Drop that enhances machine learning generalization.

Contribution

The paper presents a novel quantum-inspired divergence measure (QIF) with improved stability and theoretical bounds, and a new regularization method (QR-Drop) leveraging QIF for better ML model generalization.

Findings

QIF is more numerically stable than KL divergence in high-dimensional, partial, or disjoint support scenarios.

QR-Drop regularization improves generalization and reduces overfitting in machine learning models.

Empirical results outperform state-of-the-art methods in relevant tasks.

Abstract

Kullback--Leibler (KL) divergence is a fundamental measure of the dissimilarity between two probability distributions, but it can become unstable in high-dimensional settings due to its sensitivity to mismatches in distributional support. To address robustness limitations, we propose a novel Quantum-Inspired Fidelity-based Divergence (QIF), leveraging quantum information principles yet efficiently computable on classical hardware. Compared to KL divergence, QIF demonstrates improved numerical stability under partial or near-disjoint support conditions, thereby reducing the need for extensive regularization in specific scenarios. Moreover, QIF admits well-defined theoretical bounds and continuous similarity measures. Building on this, we introduce a novel regularization method, QR-Drop, which utilizes QIF to improve generalization in machine learning models. Empirical results show that QR-Drop effectively mitigates overfitting and outperforms state-of-the-art methods.