Data heterogeneity's impact on the performance of frequent itemset mining algorithms

• Published in: Information sciences Vol. 678; p. 120981
• Main Authors: Trasierras, Antonio Manuel, Luna, José María, Fournier-Viger, Philippe, Ventura, Sebastián
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.09.2024
• Subjects: Data heterogeneity; Frequent itemset mining; Hypercube decomposition; Itemset mining performance; k-items machine; Frequent itemset mining; Data heterogeneity; Itemset mining performance; Hypercube decomposition; k-items machine
• ISSN: 0020-0255; 1872-6291
• DOI: 10.1016/j.ins.2024.120981

Frequent itemset mining (FIM) is a widely used task that extracts frequently occurring itemsets from data. Plenty of deterministic algorithms are available for this daunting task. However, experimental studies have not considered that data heterogeneity significantly impacts the algorithms' performance, giving rise to unfair comparisons and biased conclusions. This paper seeks to advance by comparing cutting-edge algorithms using various frequency thresholds, considering the resulting data heterogeneity. An extensive experimental study is carried out, including the number of itemsets mined per second as the performance quality measure to compare algorithms. The experiments include defining eight metrics to quantify data heterogeneity, and their values vary the algorithms' performance. The results revealed that some techniques (hypercube decomposition and k-items machine) are essential to achieve excellent performance on any dataset, and most algorithms behave similarly well when they include those techniques. As a final important point, different threshold values produce dissimilar data subsets (data heterogeneity is not an immutable data characteristic), so a previous study on the database characteristics with a few minimum support thresholds could be beneficial to select the best-suited FIM algorithm beforehand.