An efficient frequent patterns mining algorithm based on MaPreduce framework

• Published in: International Conference on Software Intelligence Technologies and Applications & International Conference on Frontiers of Internet of Things 2014 pp. 1 - 5
• Main Authors: Run-Ming Yu, Ming-Gong Lee, Yuan-Shao Huang, Shi-Xuan Chen
• Format: Conference Proceeding
• Language: English
• Published: Stevenage, UK IET 2014; The Institution of Engineering & Technology
• Subjects: Business and administrative computing; Data handling techniques; Internet software; Knowledge engineering techniques; Parallel software; FAMR; MapReduce operation; data mining; Big Data; Apriori algorithm; Hadoop MapReduce; cloud computing; parallel processing; frequent pattern mining algorithm based on MapReduce framework; business data processing; one-phase algorithm
• ISBN: 9781849199704; 1849199701
• DOI: 10.1049/cp.2014.1525

Recently, data collected from business have continuously growing in every enterprise. The Big Data, Cloud Computing, Data Mining has become hot topics at the present day. How to acquire important information quickly from these data is a critical issue. In this paper, we modified the traditional Apriori algorithm by improving the execution efficiency, since Aprori algorithm has confronted with a drawback that the computation time increases dramatically when data size increases. Since the one-phase algorithm only used one MapReduce operation, it will generate excessive candidates and result in insufficient memory. We design and implement an efficient algorithm: Frequent Patterns Mining Algorithm Based on MapReduce Framework (FAMR). We adopt Hadoop MapReduce as the experiment platform. The experiment results have shown that FAMR has 16.2 speedup at last in the running time compared with one-phase algorithm.