A scalable association rule learning heuristic for large datasets

• Published in: Journal of big data Vol. 8; no. 1; pp. 1 - 32
• Main Authors: Li, Haosong, Sheu, Phillip C.-Y.
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 09.06.2021; Springer Nature B.V; SpringerOpen
• Subjects: Algorithms; Apriori algorithm; Association rule learning; Big Data; Communications Engineering; Complexity; Computational Science and Engineering; Computer Science; Data Mining and Knowledge Discovery; Database Management; Datasets; FP-Growth algorithm; Frequent itemset mining; Graph partitioning; Heuristic; Heuristic methods; Information Storage and Retrieval; Machine learning; Mathematical Applications in Computer Science; Networks; Scalability; Graph partitioning; Frequent itemset mining; Apriori algorithm; Association rule learning; Scalability; FP-Growth algorithm
• ISSN: 2196-1115; 2196-1115
• DOI: 10.1186/s40537-021-00473-3

Many algorithms have proposed to solve the association rule learning problem. However, most of these algorithms suffer from the problem of scalability either because of tremendous time complexity or memory usage, especially when the dataset is large and the minimum support ( minsup ) is set to a lower number. This paper introduces a heuristic approach based on divide-and-conquer which may exponentially reduce both the time complexity and memory usage to obtain approximate results that are close to the accurate results. It is shown from comparative experiments that the proposed heuristic approach can achieve significant speedup over existing algorithms.