Finding and Tracking Multi-Density Clusters in Online Dynamic Data Streams

• Published in: IEEE transactions on big data Vol. 8; no. 1; pp. 178 - 192
• Main Authors: Fahy, Conor, Yang, Shengxiang
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.02.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Aging; Algorithms; Big Data; Buffers; change detection; Clustering; Clustering algorithms; concept drift; concept evolution; Data mining; Data stream clustering; Data transmission; Density; Heuristic algorithms; multi-density clustering; Outliers (statistics); Particle swarm optimization; Real-time systems; Swarm intelligence; Tracking
• ISSN: 2332-7790; 2372-2096
• DOI: 10.1109/TBDATA.2019.2922969

Change is one of the biggest challenges in dynamic stream mining. From a data-mining perspective, adapting and tracking change is desirable in order to understand how and why change has occurred. Clustering, a form of unsupervised learning, can be used to identify the underlying patterns in a stream. Density-based clustering identifies clusters as areas of high density separated by areas of low density. This paper proposes a Multi-Density Stream Clustering (MDSC) algorithm to address these two problems; the multi-density problem and the problem of discovering and tracking changes in a dynamic stream. MDSC consists of two on-line components; discovered, labelled clusters and an outlier buffer. Incoming points are assigned to a live cluster or passed to the outlier buffer. New clusters are discovered in the buffer using an ant-inspired swarm intelligence approach. The newly discovered cluster is uniquely labelled and added to the set of live clusters. Processed data is subject to an ageing function and will disappear when it is no longer relevant. MDSC is shown to perform favourably to state-of-the-art peer stream-clustering algorithms on a range of real and synthetic data-streams. Experimental results suggest that MDSC can discover qualitatively useful patterns while being scalable and robust to noise.