Distributed database data importing method and system

• Main Authors: CHEN CONG, ZHANG LEI, TIE JINCHENG, YAN PINGPING, JIA LONGLONG
• Format: Patent
• Language: Chinese; English
• Published: 28.07.2023
• Subjects: CALCULATING; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS

The invention relates to a distributed database data import method. The method comprises the following steps: creating a file storage main task; based on the file storage main task, calling a file processing node to search for an original data file from a storage module, and determining the number of fragments of the original data file; slicing the original data file according to the fragment number to obtain a plurality of small files; creating fragmentation tasks corresponding to the fragmentation number, and sending the fragmentation tasks to a RabbitMQ message queue; the data storage node monitors the fragmentation task in the RabbitMQ message queue, executes the fragmentation task and updates the state of the fragmentation task; and checking the fragmentation task states of the plurality of fragmentation tasks under the file storage main task, and updating the state of the file storage main task. Compared with the prior art, the method has the advantages that a multi-node parallel fragmentation processin