Semi-Supervised Nearest Mean Classification Through a Constrained Log-Likelihood

• Published in: IEEE transaction on neural networks and learning systems Vol. 26; no. 5; pp. 995 - 1006
• Main Authors: Loog, Marco, Jensen, Are Charles
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.05.2015
• Subjects: Constrained estimation; Equations; Error analysis; Estimation; Gaussian distribution; inherent loss; log-likelihood; nearest centroid; nearest mean classifier; semi-supervised learning; Semisupervised learning; surrogate loss; Training; surrogate loss; inherent loss; Constrained estimation; log-likelihood; semi-supervised learning; nearest centroid; nearest mean classifier
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2014.2329567

We cast a semi-supervised nearest mean classifier, previously introduced by the first author, in a more principled log-likelihood formulation that is subject to constraints. This, in turn, leads us to make the important suggestion to not only investigate error rates of semi-supervised learners but also consider the risk they originally aim to optimize. We demonstrate empirically that in terms of classification error, mixed results are obtained when comparing supervised to semi-supervised nearest mean classification, while in terms of log-likelihood on the test set, the semi-supervised method consistently outperforms its supervised counterpart. Comparisons to self-learning, a standard approach in semi-supervised learning, are included to further clarify the way, in which our constrained nearest mean classifier improves over regular, supervised nearest mean classification.