Multi-objective learning of Relevance Vector Machine classifiers with multi-resolution kernels

• Published in: Pattern recognition Vol. 45; no. 9; pp. 3535 - 3543
• Main Authors: Clark, Andrew R.J., Everson, Richard M.
• Format: Journal Article
• Language: English
• Published: Kidlington Elsevier Ltd 01.09.2012; Elsevier
• Subjects: Applied sciences; Classification; Classifiers; Complexity; Cross-validation; Evolutionary algorithm; Exact sciences and technology; Information, signal and communications theory; Kernels; Mathematical analysis; Multi-resolution kernels; Optimization; Pattern recognition; Relevance Vector Machine; Signal and communications theory; Signal representation. Spectral analysis; Signal, noise; Telecommunications and information theory; Tradeoffs; Vectors (mathematics); Evolutionary algorithm; Cross-validation; Multi-resolution kernels; Classification; Relevance Vector Machine; Automatic classification; False positive; Pareto optimum; Evolutionary computation; Multiobjective programming; Cross validation; Vector method; Signal classification; Multiresolution analysis; Kernel method; Learning; Accuracy; Relevance criterion
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2012.02.025

The Relevance Vector Machine (RVM) is a sparse classifier in which complexity is controlled with the Automatic Relevance Determination prior. However, sparsity is dependent on kernel choice and severe over-fitting can occur. We describe multi-objective evolutionary algorithms (MOEAs) which optimise RVMs allowing selection of the best operating true and false positive rates and complexity from the Pareto set of optimal trade-offs. We introduce several cross-validation methods for use during evolutionary optimisation. Comparisons on benchmark datasets using multi-resolution kernels show that the MOEAs can locate markedly sparser RVMs than the standard, with comparable accuracies. ► The RVM is a sparse classifier which uses ARD to control complexity. ► Kernel choice can lead to severe overfitting. ► Multi-objective evolutionary algorithms provide Pareto set of optimal trade-offs. ► Objectives are true positive rate, false positive rate and complexity. ► K-fold cross-validation during optimisation effectively controls over-fitting.