Parallel TID-based frequent pattern mining algorithm on a PC Cluster and grid computing system

• Published in: Expert systems with applications Vol. 37; no. 3; pp. 2486 - 2494
• Main Authors: Yu, Kun-Ming, Zhou, Jiayi
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 15.03.2010
• Subjects: Association rules; Cluster computing; Data mining; Frequent pattern mining; Grid computing; Frequent pattern mining; Grid computing; Association rules; Data mining; Cluster computing
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2009.07.072

The mining of frequent patterns from transaction-oriented databases is an important subject. Frequent patterns are fundamental in generating association rules, time series, etc. Most frequent pattern mining algorithms can be classified into two categories: generate-and-test approach (Apriori-like) and pattern growth approach (FP-tree). In recent years, many techniques have been proposed for frequent pattern mining based on the FP-tree approach since it only needs two database scans. However, for pattern growth methods, the execution time increases rapidly when the database size increases or when the given support is small. Therefore, parallel-distributed computing is a good strategy for solving this problem. Some parallel algorithms have been proposed, but the execution time is still costly when the database size is large. In this paper, two parallel mining algorithms are proposed; Tidset-based Parallel FP-tree ( TPFP-tree) and Balanced Tidset-based Parallel FP-tree ( BTP-tree) for frequent pattern mining on PC Clusters and multi-cluster grids. In order to exchange transactions efficiently, a transaction identification set ( Tidset) was used to directly select transactions instead of scanning the database. Since a Grid system is a heterogeneous computing environment, the proposed BTP-tree can balance the loading according to the computing ability of the processors. BTP-tree, TPFP-tree and PFP-tree were implemented, and datasets generated with an IBM Quest Synthetic Data Generator were used to verify the performance of TPFP-tree and BTP-tree. The experimental results showed that the TPFP-tree needed less execution time on a PC Cluster than the PFP-tree when the database increased. Moreover, the BTP-tree shortened the execution time significantly and had a better load balance capability than both the TPFP-tree and PFP-tree on a multi-cluster grid.