A new scalable parallel DBSCAN algorithm using the disjoint-set data structure

• Published in: 2012 International Conference for High Performance Computing, Networking, Storage and Analysis pp. 1 - 11
• Main Authors: Patwary, Md. Mostofa Ali, Palsetia, Diana, Agrawal, Ankit, Liao, Wei-keng, Manne, Fredrik, Choudhary, Alok
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.11.2012
• Subjects: Clustering algorithms; Data structures; Density based clustering; Disjoint-set data structure; Instruction sets; Merging; Noise; Partitioning algorithms; Union-Find algorithm; Vegetation
• ISBN: 1467308056; 9781467308052
• ISSN: 2167-4329
• DOI: 10.1109/SC.2012.9

DBSCAN is a well-known density based clustering algorithm capable of discovering arbitrary shaped clusters and eliminating noise data. However, parallelization of Dbscan is challenging as it exhibits an inherent sequential data access order. Moreover, existing parallel implementations adopt a master-slave strategy which can easily cause an unbalanced workload and hence result in low parallel efficiency. We present a new parallel Dbscan algorithm (Pdsdbscan) using graph algorithmic concepts. More specifically, we employ the disjoint-set data structure to break the access sequentiality of Dbscan. In addition, we use a tree-based bottom-up approach to construct the clusters. This yields a better-balanced workload distribution. We implement the algorithm both for shared and for distributed memory. Using data sets containing up to several hundred million high-dimensional points, we show that Pdsdbscan significantly outperforms the master-slave approach, achieving speedups up to 25.97 using 40 cores on shared memory architecture, and speedups up to 5,765 using 8,192 cores on distributed memory architecture.