Long pattern estimation by conditional random field in data stream

• Published in: 2016 3rd International Conference on Systems and Informatics (ICSAI) pp. 1013 - 1017
• Main Authors: Qinhua Huang, Weimin Ouyang
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.11.2016
• Subjects: Computational modeling; Data mining; Estimation; Hidden Markov models; Itemsets; patterns; sequence
• DOI: 10.1109/ICSAI.2016.7811099

Long sequential pattern mining is of great importance. Many different algorithms have been proposed to get accurate and time-costing result. Under the background of big data, we consider the trade-off between the accuracy demand and fast estimation priority of mining result. Thus in this paper we proposed a method combining Conditional Random Field technique with pattern mining in order to get less accurate but more fast result. CRF has natural property of handling sequence problem. We carefully researched the model of CRF in sequence application and construct our stream model. To reduce the burden of heavy cost of CRF computation, combing with the features of stream mining model, we proposed a pruning method based the item duration. Our study shows that the estimation method can be effectively and fast under big data stream framework.