On Clustering Validation Techniques

• Published in: Journal of intelligent information systems Vol. 17; no. 2-3; pp. 107 - 145
• Main Authors: Halkidi, Maria, Batistakis, Yannis, Vazirgiannis, Michalis
• Format: Journal Article
• Language: English
• Published: New York Springer Nature B.V 01.12.2001
• Subjects: Algorithms; Biology; Classification; Cluster analysis; Clustering; Customers; Data compression; Data mining; Datasets; Hypotheses; Hypothesis testing; Information systems; Learning; Shopping; Social sciences; Spatial data; Studies; Validity
• ISSN: 0925-9902; 1573-7675
• DOI: 10.1023/A:1012801612483

Cluster analysis aims at identifying groups of similar objects and, therefore helps to discover distribution of patterns and interesting correlations in large data sets. It has been subject of wide research since it arises in many application domains in engineering, business and social sciences. Especially, in the last years the availability of huge transactional and experimental data sets and the arising requirements for data mining created needs for clustering algorithms that scale and can be applied in diverse domains. This paper introduces the fundamental concepts of clustering while it surveys the widely known clustering algorithms in a comparative way. Moreover, it addresses an important issue of clustering process regarding the quality assessment of the clustering results. This is also related to the inherent features of the data set under concern. A review of clustering validity measures and approaches available in the literature is presented. Furthermore, the paper illustrates the issues that are under-addressed by the recent algorithms and gives the trends in clustering process. [PUBLICATION ABSTRACT]