A method to detect data outliers from smart urban spaces via tensor analysis

• Published in: Future generation computer systems Vol. 92; pp. 290 - 301
• Main Authors: Souza, Thiago I.A., Aquino, Andre L.L., Gomes, Danielo G.
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.03.2019
• Subjects: Multiway analysis; Outliers detection; Smart cities; Urban computing; 99-00; Urban computing; Smart cities; 00-01; Multiway analysis; Outliers detection
• ISSN: 0167-739X; 1872-7115
• DOI: 10.1016/j.future.2018.09.062

With the increasing amount of data available nowadays, especially in urban spaces, it has become critical extracting knowledge to get insight from all this big data. This need becomes even more important and less obvious to supply when these data have discrepant events (i.e., outliers). Here we propose a method to explore the multiway nature of urban spaces data in outliers detection which includes three stages: (i) dimensionality reduction, where we model data as a 3rd-order tensor; from this reduction, we extract a set of latent factors to obtain the best fit for the next classification step; (ii) classification of latent factors, where the latent factors from the stage (i) are used to generate instances of similar events in monitoring smart urban spaces which result in high-quality clusters from the factorization; and (iii) combining steps (i) and (ii) to generate a refined urban space pattern identification model. We analyzed a real large-scale dataset with valuable data captured and streamed by urban sensors from 4 cities: Elda and Rois (Spain), Nuremberg (Germany), and Tallinn (Estonia). Our results allow us to conclude there is a kind of cyclic time patterns of urban sensing. •A new method to explore the multiway nature of urban spaces data in outliers detection.•Our method outperforms the MPCA-based classical method by about 23.5% in accuracy.•We used a real large-scale dataset collected from urban sensors.•Deep understanding of the dynamics patterns from smart urban spaces.