Data stream processing method and system and computer equipment

• Main Authors: LIU JIANHAO, ZHAO SONGGENG, ZHAO SHAOLING
• Format: Patent
• Language: Chinese; English
• Published: 06.01.2023
• Subjects: CALCULATING; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS

The invention discloses a data stream processing method and system and computer equipment, and relates to the technical field of data processing.The method comprises the steps that function tasks are distributed to service data streams; carrying out parameter loading on each function task; judging whether all the function tasks are completed or not; starting multi-thread sub-task allocation execution for the uncompleted function task until all the sub-tasks are completed; and cleaning and processing the service data flow and then inserting the service data flow into a database. Through the method provided by the invention, the full-flow processing of the business data flow from the beginning of collection to the end of storage can be realized, and the development cost can be greatly reduced. 本发明公开了一种数据流处理方法、系统及计算机设备，涉及数据处理技术领域，其中，所述方法包括：对业务数据流进行功能任务分配；对各所述功能任务进行参数装载；判断各所述功能任务是否全部完成；对未完成的所述功能任务启动多线程分配子任务执行，直至所述子任务全部完成；对所述业务数据流进行清洗及处理后插入数据库中。通过本发明的方法可实现业务数据流从开始收集到入库结束的全流程处理，可大幅降低开发成本。