Sample Truncation Strategies for Outlier Removal in Geochemical Data: The MCD Robust Distance Approach Versus t-SNE Ensemble Clustering

• Published in: Mathematical geosciences Vol. 53; no. 1; pp. 105 - 130
• Main Authors: Leung, Raymond, Balamurali, Mehala, Melkumyan, Arman
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 2021; Springer Nature B.V
• Subjects: Chemistry and Earth Sciences; Chi-square test; Clustering; Computer Science; Covariance; Data; Distance; Distribution functions; Earth and Environmental Science; Earth Sciences; Embedding; Geochemistry; Geotechnical Engineering & Applied Earth Sciences; Hydrogeology; Model accuracy; Multivariate analysis; Outliers (statistics); Physics; Removal; Robustness; Statistics for Engineering; Stochasticity; Outlier detection; Geochemical data; Robust distance; Subdomain identification; Sample truncation strategies
• ISSN: 1874-8961; 1874-8953
• DOI: 10.1007/s11004-019-09839-z

The presence of outliers in geochemical data can impact the accuracy of grade models and influence the interpretation of mine assay data. Removal of outliers is therefore an important consideration in grade estimation work. This paper presents two sample truncation strategies which have been devised to reject outliers in multivariate geochemical data. In essence, a data-dependent threshold is applied to the robust distances of sorted samples to discard outliers within a given class. For robust distances based on the minimum covariance determinant (MCD) where sample deviations from the cluster centre are computed using robust estimates, the inverse chi-square cumulative distribution function is often used to compute the cutoff point, χ 1 - α , ν , under the assumption of multivariate normality. In this work, it has been observed that this approach consistently underestimates the true extent of outliers. The proposed alternatives consist of a geometric and an analytic approach. The former defines the sample truncation point as the knee of the robust distance curve in an approximately chi-square-distributed quantile–quantile plot. The latter uses the silhouette and likelihood functions to consider the degree of cohesion in the resultant inlier/outlier clusters. Both techniques significantly reduce the scatter amongst the samples retained in each domain/class. For validation, ensemble clustering based on t-distributed stochastic neighbour embedding (t-SNE) is used to study the outlier recall rate, the effects of feature selection, and spatial correlation with MCD-based outlier rejection. Visual and quantitative analyses show that the proposed methods are superior to the baseline method which rejects samples using chi-square critical values. Graphic Abstract