Novel Approach for Network Traffic Pattern Analysis using Clustering-based Collective Anomaly Detection

• Published in: Annals of data science Vol. 2; no. 1; pp. 111 - 130
• Main Authors: Ahmed, Mohiuddin, Mahmood, Abdun Naser
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.03.2015; Springer Nature B.V
• Subjects: Abnormalities; Anomalies; Artificial Intelligence; Business and Management; Clustering; Communications traffic; Computer networks; Cybersecurity; Data mining; Datasets; Denial of service attacks; Economics; Empirical analysis; False alarms; Finance; Fraud; Gene expression; Insurance; Management; Pattern analysis; Statistics for Business; Traffic flow; Clustering; Traffic analysis; Computer security; Collective anomaly detection
• ISSN: 2198-5804; 2198-5812
• DOI: 10.1007/s40745-015-0035-y

There is increasing interest in the data mining and network management communities in improving existing techniques for the prompt analysis of underlying traffic patterns. Anomaly detection is one such technique for detecting abnormalities in many different domains, such as computer network intrusion, gene expression analysis, financial fraud detection and many more. Clustering is a useful unsupervised method for both identifying underlying patterns in data and anomaly detection. However, existing clustering-based techniques have high false alarm rates and consider only individual data instances for anomaly detection. Interestingly, there are traffic flows which seem legitimate but are targeted at disrupting a normal computing environment, such as the Denial of Service (DoS) attack. The presence of such anomalous data instances explains the poor performances of existing clustering-based anomaly detection techniques. In this paper, we formulate the problem of detecting DoS attacks as a collective anomaly which is a pattern in the data when a group of similar data instances behave anomalously with respect to the entire dataset. We propose a framework for collective anomaly detection using a partitional clustering technique to detect anomalies based on an empirical analysis of an attack’s characteristics. We validate our approach by comparing its results with those from existing techniques using benchmark datasets.