Are we certain it's anomalous?
The progress in modelling time series and, more generally, sequences of structured data has recently revamped research in anomaly detection. The task stands for identifying abnormal behaviors in financial series, IT systems, aerospace measurements, and the medical domain, where anomaly detection may...
Saved in:
Main Authors | , , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
16.11.2022
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | The progress in modelling time series and, more generally, sequences of
structured data has recently revamped research in anomaly detection. The task
stands for identifying abnormal behaviors in financial series, IT systems,
aerospace measurements, and the medical domain, where anomaly detection may aid
in isolating cases of depression and attend the elderly. Anomaly detection in
time series is a complex task since anomalies are rare due to highly non-linear
temporal correlations and since the definition of anomalous is sometimes
subjective. Here we propose the novel use of Hyperbolic uncertainty for Anomaly
Detection (HypAD). HypAD learns self-supervisedly to reconstruct the input
signal. We adopt best practices from the state-of-the-art to encode the
sequence by an LSTM, jointly learned with a decoder to reconstruct the signal,
with the aid of GAN critics. Uncertainty is estimated end-to-end by means of a
hyperbolic neural network. By using uncertainty, HypAD may assess whether it is
certain about the input signal but it fails to reconstruct it because this is
anomalous; or whether the reconstruction error does not necessarily imply
anomaly, as the model is uncertain, e.g. a complex but regular input signal.
The novel key idea is that a detectable anomaly is one where the model is
certain but it predicts wrongly. HypAD outperforms the current state-of-the-art
for univariate anomaly detection on established benchmarks based on data from
NASA, Yahoo, Numenta, Amazon, and Twitter. It also yields state-of-the-art
performance on a multivariate dataset of anomaly activities in elderly home
residences, and it outperforms the baseline on SWaT. Overall, HypAD yields the
lowest false alarms at the best performance rate, thanks to successfully
identifying detectable anomalies. |
---|---|
AbstractList | The progress in modelling time series and, more generally, sequences of
structured data has recently revamped research in anomaly detection. The task
stands for identifying abnormal behaviors in financial series, IT systems,
aerospace measurements, and the medical domain, where anomaly detection may aid
in isolating cases of depression and attend the elderly. Anomaly detection in
time series is a complex task since anomalies are rare due to highly non-linear
temporal correlations and since the definition of anomalous is sometimes
subjective. Here we propose the novel use of Hyperbolic uncertainty for Anomaly
Detection (HypAD). HypAD learns self-supervisedly to reconstruct the input
signal. We adopt best practices from the state-of-the-art to encode the
sequence by an LSTM, jointly learned with a decoder to reconstruct the signal,
with the aid of GAN critics. Uncertainty is estimated end-to-end by means of a
hyperbolic neural network. By using uncertainty, HypAD may assess whether it is
certain about the input signal but it fails to reconstruct it because this is
anomalous; or whether the reconstruction error does not necessarily imply
anomaly, as the model is uncertain, e.g. a complex but regular input signal.
The novel key idea is that a detectable anomaly is one where the model is
certain but it predicts wrongly. HypAD outperforms the current state-of-the-art
for univariate anomaly detection on established benchmarks based on data from
NASA, Yahoo, Numenta, Amazon, and Twitter. It also yields state-of-the-art
performance on a multivariate dataset of anomaly activities in elderly home
residences, and it outperforms the baseline on SWaT. Overall, HypAD yields the
lowest false alarms at the best performance rate, thanks to successfully
identifying detectable anomalies. |
Author | Podo, Luca Prenkaj, Bardh Aragona, Dario Flaborea, Alessandro Munjal, Bharti Galasso, Fabio Sterpa, Marco Aurelio |
Author_xml | – sequence: 1 givenname: Alessandro surname: Flaborea fullname: Flaborea, Alessandro – sequence: 2 givenname: Bardh surname: Prenkaj fullname: Prenkaj, Bardh – sequence: 3 givenname: Bharti surname: Munjal fullname: Munjal, Bharti – sequence: 4 givenname: Marco Aurelio surname: Sterpa fullname: Sterpa, Marco Aurelio – sequence: 5 givenname: Dario surname: Aragona fullname: Aragona, Dario – sequence: 6 givenname: Luca surname: Podo fullname: Podo, Luca – sequence: 7 givenname: Fabio surname: Galasso fullname: Galasso, Fabio |
BackLink | https://doi.org/10.48550/arXiv.2211.09224$$DView paper in arXiv |
BookMark | eNotzrtug0AQQNEtksIh-QA3Nl0q8Ozsg6WKEIofElIa92iAWQnJhmhxXn9vmbi63dV5Eg_DOLAQSwmpdsbAhsJv_50iSplCjqgXYlUEjn84bjlcqB_i_vI6xTSMZzqNX9Pbs3j0dJr45d5IHLfvx3KfVB-7Q1lUCdlMJ43mFqxhSx7AkJe6aZga1N5m6GQufaYBGdiA9U4p2XWA2BI65XLoSEVi_b-dgfVn6M8U_uobtJ6h6goYwjgU |
ContentType | Journal Article |
Copyright | http://arxiv.org/licenses/nonexclusive-distrib/1.0 |
Copyright_xml | – notice: http://arxiv.org/licenses/nonexclusive-distrib/1.0 |
DBID | AKY GOX |
DOI | 10.48550/arxiv.2211.09224 |
DatabaseName | arXiv Computer Science arXiv.org |
DatabaseTitleList | |
Database_xml | – sequence: 1 dbid: GOX name: arXiv.org url: http://arxiv.org/find sourceTypes: Open Access Repository |
DeliveryMethod | fulltext_linktorsrc |
ExternalDocumentID | 2211_09224 |
GroupedDBID | AKY GOX |
ID | FETCH-LOGICAL-a674-b4ec065e6af005af14bbeab24f6728191f7402e0e506f8331dd022ca283890da3 |
IEDL.DBID | GOX |
IngestDate | Mon Jan 08 05:40:21 EST 2024 |
IsDoiOpenAccess | true |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-a674-b4ec065e6af005af14bbeab24f6728191f7402e0e506f8331dd022ca283890da3 |
OpenAccessLink | https://arxiv.org/abs/2211.09224 |
ParticipantIDs | arxiv_primary_2211_09224 |
PublicationCentury | 2000 |
PublicationDate | 2022-11-16 |
PublicationDateYYYYMMDD | 2022-11-16 |
PublicationDate_xml | – month: 11 year: 2022 text: 2022-11-16 day: 16 |
PublicationDecade | 2020 |
PublicationYear | 2022 |
Score | 1.8663713 |
SecondaryResourceType | preprint |
Snippet | The progress in modelling time series and, more generally, sequences of
structured data has recently revamped research in anomaly detection. The task
stands... |
SourceID | arxiv |
SourceType | Open Access Repository |
SubjectTerms | Computer Science - Computer Vision and Pattern Recognition Computer Science - Learning |
Title | Are we certain it's anomalous? |
URI | https://arxiv.org/abs/2211.09224 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwdV1NS8QwEB129-RFFJX1a8lB8BRs0zRtTrKI6yKolxV6K5N2Agu6Sndd_flO04pevCZDYDIDbx4zeQG4MC6tUm-8tKRIau0iaZO6lpkjnUY-dnGI9MOjmT_r-yItBiB-3sJg87XcdvrAbn2lVKuwaRlmhjBUqh3ZunsquuZkkOLq7X_tuMYMS39AYrYHu311J6ZdOPZhQKsDmEwbEp8kqq77Lpaby7XA1dsrvjDtvj6Exex2cTOX_bcEEk2mpdNUMW6TQc8ZjD7WzhE6pb3J2q5U7DPmZBRRGhmfJ0lc14yTFTKO5zaqMTmCETN7GoOwOsG8Ui7jIoFpksLaW0TjlY-x4hOPYRycKd875Ymy9bMMfp78v3UKO6qd0W9n1cwZjDbNB50zcm7cJFzfN-2abD8 |
link.rule.ids | 228,230,783,888 |
linkProvider | Cornell University |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Are+we+certain+it%27s+anomalous%3F&rft.au=Flaborea%2C+Alessandro&rft.au=Prenkaj%2C+Bardh&rft.au=Munjal%2C+Bharti&rft.au=Sterpa%2C+Marco+Aurelio&rft.date=2022-11-16&rft_id=info:doi/10.48550%2Farxiv.2211.09224&rft.externalDocID=2211_09224 |