Interpretable clustering: an optimization approach

State-of-the-art clustering algorithms provide little insight into the rationale for cluster membership, limiting their interpretability. In complex real-world applications, the latter poses a barrier to machine learning adoption when experts are asked to provide detailed explanations of their algor...

Full description

Saved in:
Bibliographic Details
Published inMachine learning Vol. 110; no. 1; pp. 89 - 138
Main Authors Bertsimas, Dimitris, Orfanoudaki, Agni, Wiberg, Holly
Format Journal Article
LanguageEnglish
Published New York Springer US 2021
Springer Nature B.V
Subjects
Algorithms
Artificial Intelligence
Clustering
Computer Science
Control
Machine Learning
Mechatronics
Mixed integer
Natural Language Processing (NLP)
Optimization
Optimization techniques
Robotics
Simulation and Modeling
Clustering
Interpretability
Mixed integer optimization
Unsupervised learning
Online AccessGet full text

Cover

More Information
Summary:State-of-the-art clustering algorithms provide little insight into the rationale for cluster membership, limiting their interpretability. In complex real-world applications, the latter poses a barrier to machine learning adoption when experts are asked to provide detailed explanations of their algorithms’ recommendations. We present a new unsupervised learning method that leverages Mixed Integer Optimization techniques to generate interpretable tree-based clustering models. Utilizing a flexible optimization-driven framework, our algorithm approximates the globally optimal solution leading to high quality partitions of the feature space. We propose a novel method which can optimize for various clustering internal validation metrics and naturally determines the optimal number of clusters. It successfully addresses the challenge of mixed numerical and categorical data and achieves comparable or superior performance to other clustering methods on both synthetic and real-world datasets while offering significantly higher interpretability.
ISSN:0885-6125
1573-0565
DOI:10.1007/s10994-020-05896-2