Data validation and missing data reconstruction using self-organizing map for water treatment

• Published in: Neural computing & applications Vol. 20; no. 4; pp. 575 - 588
• Main Authors: Lamrini, B., Lakhal, El-K., Le Lann, M-V., Wehenkel, L.
• Format: Journal Article Web Resource
• Language: English
• Published: London Springer-Verlag 01.06.2011; Springer; Springer Science & Business Media B.V
• Subjects: Applied sciences; Artificial Intelligence; Calculus of variations and optimal control; Computational Biology/Bioinformatics; Computational Science and Engineering; Computer Science; Computer science; control theory; systems; Data Mining and Knowledge Discovery; Engineering, computing & technology; Exact sciences and technology; Functions of a complex variable; Image Processing and Computer Vision; Ingénierie, informatique & technologie; Learning and adaptive systems; Mathematical analysis; Mathematics; Neural networks; Numerical analysis; Numerical analysis. Scientific computation; Numerical methods in mathematical programming, optimization and calculus of variations; Numerical methods in optimization and calculus of variations; Original Article; Probability and Statistics in Computer Science; Sciences and techniques of general use; Sciences informatiques; Water treatment; Drinking water treatment; Coagulation process; Data validation; Missing data reconstruction; Self-organizing maps; Anomaly detection; Reconstruction; Visualization; Neural computation; Experimental test; Experimental data; Cluster analysis (statistics); Measurement sensor; Optimization method; Neural network; Optimization; Functional; Missing data; Treatment; Marginal distribution; Metric
• ISSN: 0941-0643; 1433-3058; 1433-3058
• DOI: 10.1007/s00521-011-0526-5

Applications in the water treatment domain generally rely on complex sensors located at remote sites. The processing of the corresponding measurements for generating higher-level information such as optimization of coagulation dosing must therefore account for possible sensor failures and imperfect input data. In this paper, self-organizing map (SOM)-based methods are applied to multiparameter data validation and missing data reconstruction in a drinking water treatment. The SOM is a special kind of artificial neural networks that can be used for analysis and visualization of large high-dimensional data sets. It performs both in a nonlinear mapping from a high-dimensional data space to a low-dimensional space aiming to preserve the most important topological and metric relationships of the original data elements and, thus, inherently clusters the data. Combining the SOM results with those obtained by a fuzzy technique that uses marginal adequacy concept to identify the functional states (normal or abnormal), the SOM performances of validation and reconstruction process are tested successfully on the experimental data stemming from a coagulation process involved in drinking water treatment.