Bad data identification method based on coupling of proximity analysis and neural network prediction

• Main Authors: WU YANGWEN, MA SHANWEI, YU YONGNING, PENG XINGWEN, QI HEMEI, SHAO QU, LIU JI
• Format: Patent
• Language: Chinese; English
• Published: 29.07.2022
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; DATA PROCESSING SYSTEMS OR METHODS, SPECIALLY ADAPTED FORADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORYOR FORECASTING PURPOSES; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE,COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTINGPURPOSES, NOT OTHERWISE PROVIDED FOR

The invention provides a bad data detection method based on coupling of proximity analysis and neural network prediction. The method mainly comprises the steps of obtaining original data, conducting dimensionless processing on the original data, calculating the average distance between a sample and a fixed number of adjacent samples, and recording the samples with the distance values smaller than a set value as bad samples. And using the pre-screened normal sample data to learn and train to establish a BP neural network model, performing loop test on the bad samples through the neural network model, and finally obtaining a bad data set. According to the invention, by coupling the proximity analysis and the neural network model, the bad data is preliminarily judged by calculating the average distance between the sample and the adjacent sample, then the bad sample is predicted by using the neural network, and the bad sample is re-identified according to the prediction result. The method is very suitable for pro