Multiple Ordinal Regression by Maximizing the Sum of Margins

• Published in: IEEE transaction on neural networks and learning systems Vol. 27; no. 10; pp. 2072 - 2083
• Main Authors: Hamsici, Onur C., Martinez, Aleix M.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.10.2016; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Algorithms; Computer vision; Decision Making; direct attribute prediction (DAP); Humans; Information retrieval; Iterative methods; Learning; Learning systems; Neural Networks, Computer; object classification; Optimization; ordinal regression; sequential minimal optimization (SMO); Support Vector Machine; support vector machine (SVM); Support vector machines; Training
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2015.2477321

Human preferences are usually measured using ordinal variables. A system whose goal is to estimate the preferences of humans and their underlying decision mechanisms requires to learn the ordering of any given sample set. We consider the solution of this ordinal regression problem using a support vector machine algorithm. Specifically, the goal is to learn a set of classifiers with common direction vectors and different biases correctly separating the ordered classes. Current algorithms are either required to solve a quadratic optimization problem, which is computationally expensive, or based on maximizing the minimum margin (i.e., a fixed-margin strategy) between a set of hyperplanes, which biases the solution to the closest margin. Another drawback of these strategies is that they are limited to order the classes using a single ranking variable (e.g., perceived length). In this paper, we define a multiple ordinal regression algorithm based on maximizing the sum of the margins between every consecutive class with respect to one or more rankings (e.g., perceived length and weight). We provide derivations of an efficient, easy-to-implement iterative solution using a sequential minimal optimization procedure. We demonstrate the accuracy of our solutions in several data sets. In addition, we provide a key application of our algorithms in estimating human subjects' ordinal classification of attribute associations to object categories. We show that these ordinal associations perform better than the binary one typically employed in the literature.