Bayesian generalized probability calculus for density matrices

• Published in: Machine learning Vol. 78; no. 1-2; pp. 63 - 101
• Main Authors: Warmuth, Manfred K., Kuzmin, Dima
• Format: Journal Article
• Language: English
• Published: Boston Springer US 2010; Springer; Springer Nature B.V
• Subjects: Artificial Intelligence; Bayesian analysis; Calculus; Computer Science; Control; Density; Exact sciences and technology; Mechatronics; Natural Language Processing (NLP); Physics; Probability; Quantum statistical mechanics; Robotics; Simulation and Modeling; Statistical physics, thermodynamics, and nonlinear dynamical systems; Probability calculus; Quantum Bayes rule; Density matrix; Generalized probability; Bayes estimation; Probabilistic approach; Entropy; Probability distribution; Variance; Probability density; Upper bound; Divergences; Diagonal matrix; Quantum mechanics; A posteriori estimation; Minimum principle; Parallel computation; Modelling; Maximum likelihood
• ISSN: 0885-6125; 1573-0565
• DOI: 10.1007/s10994-009-5133-7

One of the main concepts in quantum physics is a density matrix, which is a symmetric positive definite matrix of trace one. Finite probability distributions can be seen as a special case when the density matrix is restricted to be diagonal. We develop a probability calculus based on these more general distributions that includes definitions of joints, conditionals and formulas that relate these, including analogs of the Theorem of Total Probability and various Bayes rules for the calculation of posterior density matrices. The resulting calculus parallels the familiar “conventional” probability calculus and always retains the latter as a special case when all matrices are diagonal. We motivate both the conventional and the generalized Bayes rule with a minimum relative entropy principle, where the Kullbach-Leibler version gives the conventional Bayes rule and Umegaki’s quantum relative entropy the new Bayes rule for density matrices. Whereas the conventional Bayesian methods maintain uncertainty about which model has the highest data likelihood, the generalization maintains uncertainty about which unit direction has the largest variance. Surprisingly the bounds also generalize: as in the conventional setting we upper bound the negative log likelihood of the data by the negative log likelihood of the MAP estimator.