A novel SVM-kNN-PSO ensemble method for intrusion detection system

• Published in: Applied soft computing Vol. 38; pp. 360 - 372
• Main Authors: Aburomman, Abdulla Amin, Ibne Reaz, Mamun Bin
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.01.2016
• Subjects: Ensemble; k-NN; LUS; PSO; SVM; Weighted majority voting (WMV); k-NN; LUS; SVM; PSO; Weighted majority voting (WMV); Ensemble
• ISSN: 1568-4946; 1872-9681
• DOI: 10.1016/j.asoc.2015.10.011

The objective of this paper is to develop ensemble based classifiers that will improve the accuracy of Intrusion Detection. For this purpose, we trained and tested 12 experts and then combined them into an ensemble. We used the PSO algorithm to weight the opinion of each expert. Because the quality of the behavioral parameters inserted by the user into PSO strongly affects its effectiveness, we have used the LUS method as a meta-optimizer for finding high-quality parameters. We then used the improved PSO to create new weights for each expert. For comparison, we also developed an ensemble classifier with weights generated using WMA [12]. Fig. 1 depicts the entire process. For simplicity, the system framework was divided into the following seven stages:1.Kdd99 data pre-processing.2.Data classification with six different SVM experts.3.Data classification with six different k-NN experts.4.Data classification with ensemble classifier based on PSO.5.Data classification with ensemble classifier based on LUS improvement of PSO.6.Data classification with ensemble classifier based on WMA.7.Comparison of results for each approach. [Display omitted] •IDS implemented using ensemble of a six SVM and a six k-NN classifier.•Ensembles are created with weight generated by PSO and meta-PSO algorithms.•These two ensembles outperform third ensemble system that is created with WMA. In machine learning, a combination of classifiers, known as an ensemble classifier, often outperforms individual ones. While many ensemble approaches exist, it remains, however, a difficult task to find a suitable ensemble configuration for a particular dataset. This paper proposes a novel ensemble construction method that uses PSO generated weights to create ensemble of classifiers with better accuracy for intrusion detection. Local unimodal sampling (LUS) method is used as a meta-optimizer to find better behavioral parameters for PSO. For our empirical study, we took five random subsets from the well-known KDD99 dataset. Ensemble classifiers are created using the new approaches as well as the weighted majority algorithm (WMA) approach. Our experimental results suggest that the new approach can generate ensembles that outperform WMA in terms of classification accuracy.