Shared kernel models for class conditional density estimation

• Published in: IEEE transactions on neural networks Vol. 12; no. 5; pp. 987 - 997
• Main Authors: Titsias, M.K., Likas, A.C.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.09.2001
• Subjects: Adaptation; Algorithms; Computer science; Density; Density functional theory; Kernel; Kernels; Mathematical models; Maximum likelihood estimation; Networks; Neural networks; Pattern recognition; Probability; Probability theory; Radial basis function networks; Unsupervised learning
• ISSN: 1045-9227; 1941-0093
• DOI: 10.1109/72.950129

We present probabilistic models which are suitable for class conditional density estimation and can be regarded as shared kernel models where sharing means that each kernel may contribute to the estimation of the conditional densities of an classes. We first propose a model that constitutes an adaptation of the classical radial basis function (RBF) network (with full sharing of kernels among classes) where the outputs represent class conditional densities. In the opposite direction is the approach of separate mixtures model where the density of each class is estimated using a separate mixture density (no sharing of kernels among classes). We present a general model that allows for the expression of intermediate cases where the degree of kernel sharing can be specified through an extra model parameter. This general model encompasses both the above mentioned models as special cases. In all proposed models the training process is treated as a maximum likelihood problem and expectation-maximization algorithms have been derived for adjusting the model parameters.