Measuring Early Detection of Anomalies

• Published in: IEEE access Vol. 10; p. 1
• Main Authors: Lopez-Vizcaino, Manuel F., Novoa, Francisco J., Fernandez, Diego, Cacheda, Fidel
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Anomalies; Classification algorithms; Datasets; Early detection; Evaluation; Machine learning; Network security; Real-Time systems; Social network services; Social networks; Time-Aware metrics; Weather forecasting
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2022.3224467

Early detection is a matter of growing importance in multiple domains as network security, health conditions over social network services or weather forecasts related disasters. It is not enough to make a good decision but it also needs to be made on time. In this paper, we define a method to evaluate detection of anomalies in time-aware systems. To do so, we present the early detection problem from a generic perspective, examine the evaluation metrics available and propose a new metric, named TaP (Time aware Precision) . A set of experiments using three different datasets from different fields are performed in order to compare the behaviour of the different metrics. Two different approaches were followed, first a batch evaluation is performed, followed by a streaming evaluation which allows to present a more realistic behaviour of the systems. For both steps, we propose two sets of experiments. The first one using baseline models, followed by the evaluation of a set of Machine Learning algorithms results. The presented metric allows the amount of items needed to take a decision to be taken into account, not depending on the specific dataset but on the nature of the problem to solve.