A Short Note on the Jensen-Shannon Divergence between Simple Mixture Distributions

• Published in: arXiv.org
• Main Author: Geiger, Bernhard C
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 06.12.2018
• Subjects: Divergence
• ISSN: 2331-8422

This short note presents results about the symmetric Jensen-Shannon divergence between two discrete mixture distributions \(p_1\) and \(p_2\). Specifically, for \(i=1,2\), \(p_i\) is the mixture of a common distribution \(q\) and a distribution \(\tilde{p}_i\) with mixture proportion \(\lambda_i\). In general, \(\tilde{p}_1\neq \tilde{p}_2\) and \(\lambda_1\neq\lambda_2\). We provide experimental and theoretical insight to the behavior of the symmetric Jensen-Shannon divergence between \(p_1\) and \(p_2\) as the mixture proportions or the divergence between \(\tilde{p}_1\) and \(\tilde{p}_2\) change. We also provide insight into scenarios where the supports of the distributions \(\tilde{p}_1\), \(\tilde{p}_2\), and \(q\) do not coincide.