# A Short Note on the Jensen-Shannon Divergence between Simple Mixture Distributions

## TL;DR

This paper investigates the properties of the symmetric Jensen-Shannon divergence between two discrete mixture distributions, providing both theoretical analysis and experimental insights into how divergence varies with mixture proportions and distribution differences.

## Contribution

It offers new theoretical and experimental insights into the behavior of Jensen-Shannon divergence for mixture distributions with varying components and supports.

## Key findings

- Divergence behavior changes with mixture proportions.
- Insights into divergence when supports do not coincide.
- Theoretical bounds and experimental validation.

## Abstract

This short note presents results about the symmetric Jensen-Shannon divergence between two discrete mixture distributions $p_1$ and $p_2$. Specifically, for $i=1,2$, $p_i$ is the mixture of a common distribution $q$ and a distribution $\tilde{p}_i$ with mixture proportion $\lambda_i$. In general, $\tilde{p}_1\neq \tilde{p}_2$ and $\lambda_1\neq\lambda_2$. We provide experimental and theoretical insight to the behavior of the symmetric Jensen-Shannon divergence between $p_1$ and $p_2$ as the mixture proportions or the divergence between $\tilde{p}_1$ and $\tilde{p}_2$ change. We also provide insight into scenarios where the supports of the distributions $\tilde{p}_1$, $\tilde{p}_2$, and $q$ do not coincide.