Data stream filtering method and system

• Main Authors: CAO DI, HU GUANG, HUANG YAOXI, HE RUIBIN, WANG WEIBO, XIONG YING
• Format: Patent
• Language: Chinese; English
• Published: 29.07.2022
• Subjects: CALCULATING; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS

The invention is suitable for the technical field of data filtering, and provides a data stream filtering method and system, and the method comprises the following steps: receiving a data storage instruction, carrying out the safety detection of to-be-stored data, and carrying out the deletion filtering of the to-be-stored data when the safety detection is not passed; when the safety detection is passed, the next step is carried out; performing feature detection on the to-be-stored data, and judging whether redundant overlapping information exists in the to-be-stored data or not; when redundant overlapping information exists in the to-be-stored data, optimizing and filtering the redundant overlapping information to generate storable data, and storing the storable data; and directly storing the to-be-stored data when the redundant overlapping information does not exist in the to-be-stored data. According to the method, the redundant overlapped information is optimized and filtered to generate the storable data