Discovering outlying aspects in large datasets

• Published in: Data mining and knowledge discovery Vol. 30; no. 6; pp. 1520 - 1555
• Main Authors: Vinh, Nguyen Xuan, Chan, Jeffrey, Romano, Simone, Bailey, James, Leckie, Christopher, Ramamohanarao, Kotagiri, Pei, Jian
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.11.2016; Springer Nature B.V
• Subjects: Artificial Intelligence; Cancer; Chemistry and Earth Sciences; Computational efficiency; Computer Science; Data mining; Data Mining and Knowledge Discovery; Datasets; Feature selection; Information Storage and Retrieval; Mathematical models; Multidimensional data; Physics; Query processing; Scoring; Searching; Statistics for Engineering; Subspaces; Outlier explanation; Outlying aspects mining; Subspace selection
• ISSN: 1384-5810; 1573-756X
• DOI: 10.1007/s10618-016-0453-2

We address the problem of outlying aspects mining: given a query object and a reference multidimensional data set, how can we discover what aspects (i.e., subsets of features or subspaces) make the query object most outlying? Outlying aspects mining can be used to explain any data point of interest, which itself might be an inlier or outlier. In this paper, we investigate several open challenges faced by existing outlying aspects mining techniques and propose novel solutions, including (a) how to design effective scoring functions that are unbiased with respect to dimensionality and yet being computationally efficient, and (b) how to efficiently search through the exponentially large search space of all possible subspaces. We formalize the concept of dimensionality unbiasedness, a desirable property of outlyingness measures. We then characterize existing scoring measures as well as our novel proposed ones in terms of efficiency, dimensionality unbiasedness and interpretability. Finally, we evaluate the effectiveness of different methods for outlying aspects discovery and demonstrate the utility of our proposed approach on both large real and synthetic data sets.