A Hybrid Semi-Supervised Anomaly Detection Model for High-Dimensional Data

Anomaly detection, which aims to identify observations that deviate from a nominal sample, is a challenging task for high-dimensional data. Traditional distance-based anomaly detection methods compute the neighborhood distance between each observation and suffer from the curse of dimensionality in h...

Full description

Saved in:
Bibliographic Details
Published inComputational Intelligence and Neuroscience Vol. 2017; no. 2017; pp. 1 - 9
Main Authors Yang, Bo, Men, Aidong, Jiang, Zhuqing, Song, Hongchao
Format Journal Article
LanguageEnglish
Published Cairo, Egypt Hindawi Limiteds 01.01.2017
Hindawi Publishing Corporation
Hindawi
John Wiley & Sons, Inc
Hindawi Limited
Subjects
Algorithms
Anomalies
Artificial intelligence
Computer applications
Data mining
Datasets
Design of experiments
Detectors
Electronic data processing
False alarms
International conferences
Machine learning
Methods
Model accuracy
Neural networks
Pattern recognition
Researchers
Online AccessGet full text

Cover

More Information
Summary:Anomaly detection, which aims to identify observations that deviate from a nominal sample, is a challenging task for high-dimensional data. Traditional distance-based anomaly detection methods compute the neighborhood distance between each observation and suffer from the curse of dimensionality in high-dimensional space; for example, the distances between any pair of samples are similar and each sample may perform like an outlier. In this paper, we propose a hybrid semi-supervised anomaly detection model for high-dimensional data that consists of two parts: a deep autoencoder (DAE) and an ensemble k-nearest neighbor graphs- (K-NNG-) based anomaly detector. Benefiting from the ability of nonlinear mapping, the DAE is first trained to learn the intrinsic features of a high-dimensional dataset to represent the high-dimensional data in a more compact subspace. Several nonparametric KNN-based anomaly detectors are then built from different subsets that are randomly sampled from the whole dataset. The final prediction is made by all the anomaly detectors. The performance of the proposed method is evaluated on several real-life datasets, and the results confirm that the proposed hybrid model improves the detection accuracy and reduces the computational complexity.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
Academic Editor: Pedro Antonio Gutierrez
ISSN:1687-5265
1687-5273
DOI:10.1155/2017/8501683