Multi-parametric solution-path algorithm for instance-weighted support vector machines

• Published in: 2011 IEEE International Workshop on Machine Learning for Signal Processing pp. 1 - 6
• Main Authors: Karasuyama, M., Harada, N., Sugiyama, M., Takeuchi, I.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.09.2011
• Subjects: Computational modeling; Indexes; Machine learning; Machine learning algorithms; Support vector machines; Training; Vectors
• ISBN: 1457716216; 9781457716218
• ISSN: 1551-2541; 2378-928X
• DOI: 10.1109/MLSP.2011.6064551

An instance-weighted variant of the support vector machine (SVM) has attracted considerable attention recently since they are useful in various machine learning tasks such as non-stationary data analysis, heteroscedastic data modeling, transfer learning, learning to rank, and transduction. An important challenge in these scenarios is to overcome the computational bottleneck-instance weights often change dynamically or adaptively, and thus the weighted SVM solutions must be repeatedly computed. In this paper, we develop an algorithm that can efficiently and exactly update the weighted SVM solutions for arbitrary change of instance weights. Technically, this contribution can be regarded as an extension of the conventional solution-path algorithm for a single regularization parameter to multiple instance-weight parameters. However, this extension gives rise to a significant problem that breakpoints (at which the solution path turns) have to be identified in high-dimensional space. To facilitate this, we introduce a parametric representation of instance weights which allows us to find the breakpoints in high-dimensional space easily. Despite its simplicity, our parametrization covers various important machine learning tasks and it widens the applicability of the solution-path algorithm. Through extensive experiments on various practical applications, we demonstrate the usefulness of the proposed algorithm.