Method and device for improving input and output performance of big data processing architecture

• Main Authors: FAN JINGSONG, LU MING, MAO LINGXIAO, ZHANG YIN
• Format: Patent
• Language: Chinese; English
• Published: 28.03.2023
• Subjects: CALCULATING; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS

The invention provides a method and device for improving the input and output performance of a big data processing architecture, and the method comprises the steps: receiving write-in information corresponding to to-be-written data which needs to be written into a database, and storing the write-in information, the write-in information comprises at least one of a subject name, a field name, a data format, a database name, a serializer name, a subscription type and a database table name; serializing the to-be-written data according to the write-in information to obtain target data required by a target database, the target database being a database corresponding to the database name; and writing the serialized target data into the target database through a connection pool corresponding to the target database. According to the method in the embodiment, writing of any database in a big data processing architecture can be achieved, calculation and storage efficiency can be improved, and the requirement for massive