Maximum Margin Multi-Dimensional Classification

• Published in: IEEE transaction on neural networks and learning systems Vol. 33; no. 12; pp. 7185 - 7198
• Main Authors: Jia, Bin-Bin, Zhang, Min-Ling
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.12.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Class dependencies; Classification; Comparative studies; Covariance matrices; Heterogeneity; Labels; Machine learning; Matrix decomposition; maximum margin; Modelling; multi-dimensional classification (MDC); Objective function; Optimization; Predictive models; Quadratic programming; Regularization; Semantics; Task analysis
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2021.3084373

Multi-dimensional classification (MDC) assumes heterogeneous class spaces for each example, where class variables from different class spaces characterize semantics of the example along different dimensions. The heterogeneity of class spaces leads to incomparability of the modeling outputs from different class spaces, which is the major difficulty in designing MDC approaches. In this article, we make a first attempt toward adapting maximum margin techniques for MDC problem and a novel approach named M3MDC is proposed. Specifically, M3MDC maximizes the margins between each pair of class labels with respect to individual class variable while modeling relationship across class variables (as well as class labels within individual class variable) via covariance regularization. The resulting formulation admits convex objective function with nonlinear constraints, which can be solved via alternating optimization with quadratic programming (QP) or closed-form solution in either alternating step. Comparative studies on the most comprehensive real-world MDC datasets to date are conducted and it is shown that M3MDC achieves highly competitive performance against state-of-the-art MDC approaches.