IWApriori: An Association Rule Mining and Self-updating Method Based on Weighted Increment
The mining of association rules plays an important role in fault prediction. Many studies have shown that there is an obvious temporal and spatial correlation between the failure records of the cluster system. Therefore, most cluster system failure prediction engines are built based on causal correl...
Saved in:
Published in | 2020 21st Asia-Pacific Network Operations and Management Symposium (APNOMS) pp. 167 - 172 |
---|---|
Main Authors | , , , , , |
Format | Conference Proceeding |
Language | English |
Published |
KICS
01.09.2020
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | The mining of association rules plays an important role in fault prediction. Many studies have shown that there is an obvious temporal and spatial correlation between the failure records of the cluster system. Therefore, most cluster system failure prediction engines are built based on causal correlation analysis between log events. However, the original system log file usually contains a large number of invalid records (duplicate or non-fault related records), which makes the mining of event correlation extremely difficult and seriously affects the efficiency and accuracy of fault prediction. Therefore, this paper proposes an association rule mining and self-updating method based on weighted increment, named IWApriori (improved weighted Apriori algorithm). The method includes two important steps: 1) log preprocessing; 2) mining and updating of association rules based on improved algorithm IWApriori. This method can effectively improve the rule completeness and realize the efficient mining and updating of rules in the whole life cycle of the system. In addition, we used the real log data set Blue Gene/L to validate our method. The results show that our association rule mining method is better than other methods in terms of time performance, space performance and the effectiveness of mining rules. |
---|---|
AbstractList | The mining of association rules plays an important role in fault prediction. Many studies have shown that there is an obvious temporal and spatial correlation between the failure records of the cluster system. Therefore, most cluster system failure prediction engines are built based on causal correlation analysis between log events. However, the original system log file usually contains a large number of invalid records (duplicate or non-fault related records), which makes the mining of event correlation extremely difficult and seriously affects the efficiency and accuracy of fault prediction. Therefore, this paper proposes an association rule mining and self-updating method based on weighted increment, named IWApriori (improved weighted Apriori algorithm). The method includes two important steps: 1) log preprocessing; 2) mining and updating of association rules based on improved algorithm IWApriori. This method can effectively improve the rule completeness and realize the efficient mining and updating of rules in the whole life cycle of the system. In addition, we used the real log data set Blue Gene/L to validate our method. The results show that our association rule mining method is better than other methods in terms of time performance, space performance and the effectiveness of mining rules. |
Author | Huo, Yonghua An, Na Dong, Jing Ge, Zhongdi Yang, Yang Xie, Ping |
Author_xml | – sequence: 1 givenname: Yonghua surname: Huo fullname: Huo, Yonghua organization: The 54th Research Institute of CETC,Shijiazhuang,Hebei,China – sequence: 2 givenname: Jing surname: Dong fullname: Dong, Jing organization: State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications,Beijing,China – sequence: 3 givenname: Zhongdi surname: Ge fullname: Ge, Zhongdi organization: State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications,Beijing,China – sequence: 4 givenname: Ping surname: Xie fullname: Xie, Ping organization: The 54th Research Institute of CETC,Shijiazhuang,Hebei,China – sequence: 5 givenname: Na surname: An fullname: An, Na organization: The 54th Research Institute of CETC,Shijiazhuang,Hebei,China – sequence: 6 givenname: Yang surname: Yang fullname: Yang, Yang organization: The 54th Research Institute of CETC,Shijiazhuang,Hebei,China |
BookMark | eNotj8FOwzAQRI0EByj9Ai7mAxI2dmJ7uYUKSqSGIgqqxKUy9qa11DpVkh74eyLR04xmRiO9G3YZ20iM3WeQCokZPpTvb8t6VUCeiVSAgBSFVKj0BZuiNgaxAMilMdfsu1qXxy60XXjkZeRl37cu2CG0kX-c9sTrEEPcchs9X9G-SU5HP7ZjUtOwaz1_sj15Pq7XFLa7YfRVdB0dKA637Kqx-56mZ52wr5fnz9lrsljOq1m5SIIAOSQNFkpAgxYyJRBISHLSo1dOag3KNlILbRwWTeEyCxYp1-pHOa8ACjJGTtjd_28gos0Ic7Dd7-ZMLP8AHLZQnA |
ContentType | Conference Proceeding |
DBID | 6IE 6IL CBEJK RIE RIL |
DOI | 10.23919/APNOMS50412.2020.9236967 |
DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Xplore IEEE Proceedings Order Plans (POP All) 1998-Present |
DatabaseTitleList | |
Database_xml | – sequence: 1 dbid: RIE name: IEEE Xplore url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/ sourceTypes: Publisher |
DeliveryMethod | fulltext_linktorsrc |
EISBN | 9788995004388 899500438X |
EndPage | 172 |
ExternalDocumentID | 9236967 |
Genre | orig-research |
GrantInformation_xml | – fundername: National Key Research and Development Program of China grantid: 2019YFB2103200 funderid: 10.13039/501100012166 |
GroupedDBID | 6IE 6IL CBEJK RIE RIL |
ID | FETCH-LOGICAL-i203t-f95620f9a016290e23ec3d9d6c37706af37278c95f5c1a0a9e476b6cd6005e883 |
IEDL.DBID | RIE |
IngestDate | Thu Jun 29 18:39:04 EDT 2023 |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-i203t-f95620f9a016290e23ec3d9d6c37706af37278c95f5c1a0a9e476b6cd6005e883 |
PageCount | 6 |
ParticipantIDs | ieee_primary_9236967 |
PublicationCentury | 2000 |
PublicationDate | 2020-Sept. |
PublicationDateYYYYMMDD | 2020-09-01 |
PublicationDate_xml | – month: 09 year: 2020 text: 2020-Sept. |
PublicationDecade | 2020 |
PublicationTitle | 2020 21st Asia-Pacific Network Operations and Management Symposium (APNOMS) |
PublicationTitleAbbrev | APNOMS |
PublicationYear | 2020 |
Publisher | KICS |
Publisher_xml | – name: KICS |
Score | 1.7915431 |
Snippet | The mining of association rules plays an important role in fault prediction. Many studies have shown that there is an obvious temporal and spatial correlation... |
SourceID | ieee |
SourceType | Publisher |
StartPage | 167 |
SubjectTerms | Association Rule Mining Log Filtering self - updating |
Title | IWApriori: An Association Rule Mining and Self-updating Method Based on Weighted Increment |
URI | https://ieeexplore.ieee.org/document/9236967 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3NS8MwFA_bDuJJZRO_ieDRdlnTpo23KY4pdA7n2PAy0uQFhqMbo73415ukc37gwVt4BBJeQt5H3vv9ELriGWVCWXy7CGIv1FJ6WcbBs7TWJBZhKBx8cTpg_XH4OI2mNXS97YUBAFd8Br4dur98tZSlTZW1jTPCOIvrqJ6QoOrV2kGXrpqZd3i7Oxw8paPIIkiZwC8g_mb-D-IUZzd6eyj9XLEqF3nzyyLz5fsvMMb_bmkftb469PBwa3sOUA3yJnp9mHRX6_lyPb_B3Rx_0zx-LheAU0cGgUWu8AgW2itXtrfBSFJHI41vjUVT2MyeuHypGZvXo8ofttC4d_9y1_c23AnePCC08LSJewKiuTAuXcAJBBQkVVwxSeOYMKGpcVwSySMdyY4ggkMYs4xJZRygCJKEHqJGvszhCGEVRlxRzYAlWUgZSbRW0lzAoCMtWp06Rk2rl9mqgseYbVRy8rf4FO3as6nKtM5Qo1iXcG7sepFduAP9AACxpA4 |
link.rule.ids | 310,311,786,790,795,796,802,27956,55107 |
linkProvider | IEEE |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1bT8IwFD5BTNQnNWC8WxMfHZR161bf0GhAGRKBQHwhWy8JkQxCthd_vW2HeIkPvjVNly3nNPtOT8_5PoArlhAaC8Nv58vA8RTnTpIw6RhZaxzEnhdb-uKoS1tD73Hsj0twve6FkVLa4jNZM0N7ly_mPDepsroORiijwQZsapzHQdGttQWXtp6ZNVi92es-R33fcEjpo5-La6snfkinWOR42IXo851FwchbLc-SGn__Rcf434_ag-pXjx7qrdFnH0oyrcBre9RcLKfz5fQGNVP0zfboJZ9JFFk5CBSnAvXlTDn5wnQ36JnICkmjW41pAunVI5sx1WP9_ygyiFUYPtwP7lrOSj3BmbqYZI7SJx8XKxbroM5lWLpEciKYoJwEAaaxIjp0CTnzlc8bMY6Z9AKaUC50COTLMCQHUE7nqTwEJDyfCaKopGHiEYpDpQTXW9BtcMNXJ46gYuwyWRQEGZOVSY7_nr6A7dYg6kw67e7TCewYPxVFW6dQzpa5PNMonyXn1rkf1KinYg |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2020+21st+Asia-Pacific+Network+Operations+and+Management+Symposium+%28APNOMS%29&rft.atitle=IWApriori%3A+An+Association+Rule+Mining+and+Self-updating+Method+Based+on+Weighted+Increment&rft.au=Huo%2C+Yonghua&rft.au=Dong%2C+Jing&rft.au=Ge%2C+Zhongdi&rft.au=Xie%2C+Ping&rft.date=2020-09-01&rft.pub=KICS&rft.spage=167&rft.epage=172&rft_id=info:doi/10.23919%2FAPNOMS50412.2020.9236967&rft.externalDocID=9236967 |