A fast unsupervised preprocessing method for network monitoring

Identifying a network misuse takes days or even weeks, and network administrators usually neglect zero-day threats until a large number of malicious users exploit them. Besides, security applications, such as anomaly detection and attack mitigation systems, must apply real-time monitoring to reduce...

Full description

Saved in:
Bibliographic Details
Published inAnnales des télécommunications Vol. 74; no. 3-4; pp. 139 - 155
Main Authors Andreoni Lopez, Martin, Mattos, Diogo M. F., Duarte, Otto Carlos M. B., Pujolle, Guy
Format Journal Article
LanguageEnglish
Published Cham Springer International Publishing 01.04.2019
Springer Nature B.V
Springer
Subjects
Algorithms
Anomalies
Circuits
Classification
Communications Engineering
Communications traffic
Computer Communication Networks
Computer Science
Cybersecurity
Data processing
Engineering
Information and Communication
Information Systems and Communication Service
Machine learning
Monitoring
Networking and Internet Architecture
Networks
Preprocessing
R & D/Technology Policy
Reduction
Signal,Image and Speech Processing
Feature selection
Normalization
Network Monitoring
Data preprocessing
Machine learning
Online AccessGet full text

Cover

More Information
Summary:Identifying a network misuse takes days or even weeks, and network administrators usually neglect zero-day threats until a large number of malicious users exploit them. Besides, security applications, such as anomaly detection and attack mitigation systems, must apply real-time monitoring to reduce the impacts of security incidents. Thus, information processing time should be as small as possible to enable an effective defense against attacks. In this paper, we present a fast preprocessing method for network traffic classification based on feature correlation and feature normalization. Our proposed method couples a normalization and feature selection algorithms. We evaluate the proposed algorithms against three different datasets for eight different machine learning classification algorithms. Our proposed normalization algorithm reduces the classification error rate when compared with traditional methods. Our feature selection algorithm chooses an optimized subset of features improving accuracy by more than 11% within a 100-fold reduction in processing time when compared to traditional feature selection and feature reduction algorithms. The preprocessing method is performed in batch and streaming data, being able to detect concept-drift.
ISSN:0003-4347
1958-9395
DOI:10.1007/s12243-018-0663-2