Concept Drift–Based Intrusion Detection For Evolving Data Stream Classification In IDS: Approaches And Comparative Study

• Published in: Computer journal Vol. 67; no. 7; pp. 2529 - 2547
• Main Authors: Seth, Sugandh, Chahal, Kuljit Kaur, Singh, Gurvinder
• Format: Journal Article
• Language: English
• Published: Oxford University Press 20.07.2024
• Subjects: dynamic environments; deep learning; intrusion detection system; concept drift; stream-oriented learning; adaptive algorithms; evolving data stream; machine learning; model adaptation; online learning
• ISSN: 0010-4620; 1460-2067
• DOI: 10.1093/comjnl/bxae023

Static machine and deep learning algorithms are commonly used in intrusion detection systems (IDSs). However, their effectiveness is constrained by the evolving data distribution and the obsolescence of the static data sources used for model training. Consequently, static classifiers lose efficacy, necessitating expensive model retraining with time. The aim is to develop a dynamic and adaptable IDS that mitigates the limitations of static models, ensuring real-time threat detection and reducing the need for frequent, resource-intensive model retraining. This research proposes an approach that amalgamates the adaptive random forest (ARF) classifier with Hoeffding’s bounds and a moving average test for the early and accurate detection of network intrusions. The ARF can adapt in real time to shifting network conditions and evolving attack patterns, constantly refining its intrusion detection capabilities. Furthermore, the inclusion of Hoeffding’s bounds and the moving average test adds a dimension of statistical rigor to the system, facilitating the timely recognition of concept drift and distinguishing benign network variations from potential intrusions. The synergy of these techniques results in reduced false positives and false negatives, thereby enhancing the overall detection rate. The proposed method delivers outstanding results, with 99.95% accuracy and an impressive 99.96% recall rate on the latest CIC-IDS 2018 dataset, outperforming the results of existing approaches.