Time series joins, motifs, discords and shapelets: a unifying view that exploits the matrix profile

• Published in: Data mining and knowledge discovery Vol. 32; no. 1; pp. 83 - 123
• Main Authors: Yeh, Chin-Chia Michael, Zhu, Yan, Ulanova, Liudmila, Begum, Nurjahan, Ding, Yifei, Dau, Hoang Anh, Zimmerman, Zachary, Silva, Diego Furtado, Mueen, Abdullah, Keogh, Eamonn
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.01.2018; Springer Nature B.V
• Subjects: Algorithms; Artificial Intelligence; Bioinformatics; Chemistry and Earth Sciences; Computer Science; Data mining; Data Mining and Knowledge Discovery; Datasets; Deoxyribonucleic acid; DNA; Information Storage and Retrieval; Monitoring; Physics; Pruning; Seismology; Similarity; Statistics for Engineering; Time series; Motif discovery; Anomaly detection; Time series; Joins
• DOI: 10.1007/s10618-017-0519-9
• ISSN: 1384-5810

The last decade has seen a flurry of research on all-pairs-similarity-search (or similarity joins ) for text, DNA and a handful of other datatypes, and these systems have been applied to many diverse data mining problems. However, there has been surprisingly little progress made on similarity joins for time series subsequences . The lack of progress probably stems from the daunting nature of the problem. For even modest sized datasets the obvious nested-loop algorithm can take months, and the typical speed-up techniques in this domain (i.e., indexing, lower-bounding, triangular-inequality pruning and early abandoning) at best produce only one or two orders of magnitude speedup. In this work we introduce a novel scalable algorithm for time series subsequence all-pairs-similarity-search. For exceptionally large datasets, the algorithm can be trivially cast as an anytime algorithm and produce high-quality approximate solutions in reasonable time and/or be accelerated by a trivial porting to a GPU framework. The exact similarity join algorithm computes the answer to the time series motif and time series discord problem as a side-effect, and our algorithm incidentally provides the fastest known algorithm for both these extensively-studied problems. We demonstrate the utility of our ideas for many time series data mining problems, including motif discovery, novelty discovery, shapelet discovery, semantic segmentation, density estimation, and contrast set mining. Moreover, we demonstrate the utility of our ideas on domains as diverse as seismology, music processing, bioinformatics, human activity monitoring, electrical power-demand monitoring and medicine.