Are we certain it's anomalous?

The progress in modelling time series and, more generally, sequences of structured data has recently revamped research in anomaly detection. The task stands for identifying abnormal behaviors in financial series, IT systems, aerospace measurements, and the medical domain, where anomaly detection may...

Full description

Saved in:
Bibliographic Details
Main Authors Flaborea, Alessandro, Prenkaj, Bardh, Munjal, Bharti, Sterpa, Marco Aurelio, Aragona, Dario, Podo, Luca, Galasso, Fabio
Format Journal Article
LanguageEnglish
Published 16.11.2022
Subjects
Online AccessGet full text

Cover

Loading…
Abstract The progress in modelling time series and, more generally, sequences of structured data has recently revamped research in anomaly detection. The task stands for identifying abnormal behaviors in financial series, IT systems, aerospace measurements, and the medical domain, where anomaly detection may aid in isolating cases of depression and attend the elderly. Anomaly detection in time series is a complex task since anomalies are rare due to highly non-linear temporal correlations and since the definition of anomalous is sometimes subjective. Here we propose the novel use of Hyperbolic uncertainty for Anomaly Detection (HypAD). HypAD learns self-supervisedly to reconstruct the input signal. We adopt best practices from the state-of-the-art to encode the sequence by an LSTM, jointly learned with a decoder to reconstruct the signal, with the aid of GAN critics. Uncertainty is estimated end-to-end by means of a hyperbolic neural network. By using uncertainty, HypAD may assess whether it is certain about the input signal but it fails to reconstruct it because this is anomalous; or whether the reconstruction error does not necessarily imply anomaly, as the model is uncertain, e.g. a complex but regular input signal. The novel key idea is that a detectable anomaly is one where the model is certain but it predicts wrongly. HypAD outperforms the current state-of-the-art for univariate anomaly detection on established benchmarks based on data from NASA, Yahoo, Numenta, Amazon, and Twitter. It also yields state-of-the-art performance on a multivariate dataset of anomaly activities in elderly home residences, and it outperforms the baseline on SWaT. Overall, HypAD yields the lowest false alarms at the best performance rate, thanks to successfully identifying detectable anomalies.
AbstractList The progress in modelling time series and, more generally, sequences of structured data has recently revamped research in anomaly detection. The task stands for identifying abnormal behaviors in financial series, IT systems, aerospace measurements, and the medical domain, where anomaly detection may aid in isolating cases of depression and attend the elderly. Anomaly detection in time series is a complex task since anomalies are rare due to highly non-linear temporal correlations and since the definition of anomalous is sometimes subjective. Here we propose the novel use of Hyperbolic uncertainty for Anomaly Detection (HypAD). HypAD learns self-supervisedly to reconstruct the input signal. We adopt best practices from the state-of-the-art to encode the sequence by an LSTM, jointly learned with a decoder to reconstruct the signal, with the aid of GAN critics. Uncertainty is estimated end-to-end by means of a hyperbolic neural network. By using uncertainty, HypAD may assess whether it is certain about the input signal but it fails to reconstruct it because this is anomalous; or whether the reconstruction error does not necessarily imply anomaly, as the model is uncertain, e.g. a complex but regular input signal. The novel key idea is that a detectable anomaly is one where the model is certain but it predicts wrongly. HypAD outperforms the current state-of-the-art for univariate anomaly detection on established benchmarks based on data from NASA, Yahoo, Numenta, Amazon, and Twitter. It also yields state-of-the-art performance on a multivariate dataset of anomaly activities in elderly home residences, and it outperforms the baseline on SWaT. Overall, HypAD yields the lowest false alarms at the best performance rate, thanks to successfully identifying detectable anomalies.
Author Podo, Luca
Prenkaj, Bardh
Aragona, Dario
Flaborea, Alessandro
Munjal, Bharti
Galasso, Fabio
Sterpa, Marco Aurelio
Author_xml – sequence: 1
  givenname: Alessandro
  surname: Flaborea
  fullname: Flaborea, Alessandro
– sequence: 2
  givenname: Bardh
  surname: Prenkaj
  fullname: Prenkaj, Bardh
– sequence: 3
  givenname: Bharti
  surname: Munjal
  fullname: Munjal, Bharti
– sequence: 4
  givenname: Marco Aurelio
  surname: Sterpa
  fullname: Sterpa, Marco Aurelio
– sequence: 5
  givenname: Dario
  surname: Aragona
  fullname: Aragona, Dario
– sequence: 6
  givenname: Luca
  surname: Podo
  fullname: Podo, Luca
– sequence: 7
  givenname: Fabio
  surname: Galasso
  fullname: Galasso, Fabio
BackLink https://doi.org/10.48550/arXiv.2211.09224$$DView paper in arXiv
BookMark eNotzrtug0AQQNEtksIh-QA3Nl0q8Ozsg6WKEIofElIa92iAWQnJhmhxXn9vmbi63dV5Eg_DOLAQSwmpdsbAhsJv_50iSplCjqgXYlUEjn84bjlcqB_i_vI6xTSMZzqNX9Pbs3j0dJr45d5IHLfvx3KfVB-7Q1lUCdlMJ43mFqxhSx7AkJe6aZga1N5m6GQufaYBGdiA9U4p2XWA2BI65XLoSEVi_b-dgfVn6M8U_uobtJ6h6goYwjgU
ContentType Journal Article
Copyright http://arxiv.org/licenses/nonexclusive-distrib/1.0
Copyright_xml – notice: http://arxiv.org/licenses/nonexclusive-distrib/1.0
DBID AKY
GOX
DOI 10.48550/arxiv.2211.09224
DatabaseName arXiv Computer Science
arXiv.org
DatabaseTitleList
Database_xml – sequence: 1
  dbid: GOX
  name: arXiv.org
  url: http://arxiv.org/find
  sourceTypes: Open Access Repository
DeliveryMethod fulltext_linktorsrc
ExternalDocumentID 2211_09224
GroupedDBID AKY
GOX
ID FETCH-LOGICAL-a674-b4ec065e6af005af14bbeab24f6728191f7402e0e506f8331dd022ca283890da3
IEDL.DBID GOX
IngestDate Mon Jan 08 05:40:21 EST 2024
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a674-b4ec065e6af005af14bbeab24f6728191f7402e0e506f8331dd022ca283890da3
OpenAccessLink https://arxiv.org/abs/2211.09224
ParticipantIDs arxiv_primary_2211_09224
PublicationCentury 2000
PublicationDate 2022-11-16
PublicationDateYYYYMMDD 2022-11-16
PublicationDate_xml – month: 11
  year: 2022
  text: 2022-11-16
  day: 16
PublicationDecade 2020
PublicationYear 2022
Score 1.8663713
SecondaryResourceType preprint
Snippet The progress in modelling time series and, more generally, sequences of structured data has recently revamped research in anomaly detection. The task stands...
SourceID arxiv
SourceType Open Access Repository
SubjectTerms Computer Science - Computer Vision and Pattern Recognition
Computer Science - Learning
Title Are we certain it's anomalous?
URI https://arxiv.org/abs/2211.09224
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwdV1NS8QwEB129-RFFJX1a8lB8BRs0zRtTrKI6yKolxV6K5N2Agu6Sndd_flO04pevCZDYDIDbx4zeQG4MC6tUm-8tKRIau0iaZO6lpkjnUY-dnGI9MOjmT_r-yItBiB-3sJg87XcdvrAbn2lVKuwaRlmhjBUqh3ZunsquuZkkOLq7X_tuMYMS39AYrYHu311J6ZdOPZhQKsDmEwbEp8kqq77Lpaby7XA1dsrvjDtvj6Exex2cTOX_bcEEk2mpdNUMW6TQc8ZjD7WzhE6pb3J2q5U7DPmZBRRGhmfJ0lc14yTFTKO5zaqMTmCETN7GoOwOsG8Ui7jIoFpksLaW0TjlY-x4hOPYRycKd875Ymy9bMMfp78v3UKO6qd0W9n1cwZjDbNB50zcm7cJFzfN-2abD8
link.rule.ids 228,230,783,888
linkProvider Cornell University
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Are+we+certain+it%27s+anomalous%3F&rft.au=Flaborea%2C+Alessandro&rft.au=Prenkaj%2C+Bardh&rft.au=Munjal%2C+Bharti&rft.au=Sterpa%2C+Marco+Aurelio&rft.date=2022-11-16&rft_id=info:doi/10.48550%2Farxiv.2211.09224&rft.externalDocID=2211_09224