Data popularity acquisition method and system based on TCPDUMP and distributed message queue

• Main Authors: CHEN TINGLIANG, MAO XIAOYAO, YUAN PANFENG, CHEN TINGQUAN
• Format: Patent
• Language: Chinese; English
• Published: 22.08.2023
• Subjects: CALCULATING; COMPUTING; COUNTING; ELECTRIC COMMUNICATION TECHNIQUE; ELECTRIC DIGITAL DATA PROCESSING; ELECTRICITY; PHYSICS; SELECTING; TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHICCOMMUNICATION

The invention discloses a data popularity acquisition method based on TCPDUMP and a distributed message queue, and belongs to the technical field of databases. The method comprises the steps of creating a data source; intercepting the SQL script sent to the data source through the TCPDUMP component, and performing script assembly to obtain an assembled script; sending the assembly script to a distributed message queue; and performing data popularity collection on the data source according to the assembly script in the distributed message queue. The invention also discloses a data popularity acquisition method based on the TCPDUMP and the distributed message queue. According to the invention, the system is ensured to keep stable operation when the data volume is large. 本发明公开一种基于TCPDUMP和分布式消息队列实现的数据热度采集方法，属于数据库技术领域；该方法包括：创建数据源；通过TCPDUMP组件截取发送至数据源的SQL脚本并进行脚本组装，得到组装脚本；将组装脚本发送到分布式消息队列；根据分布式消息队列中的组装脚本，对数据源进行数据热度采集。本发明还公开一种基于TCPDUMP和分布式消息队列实现的数据热度采集方法。本发明保证了系统在大数据量的时候也能保持稳定运行。