Kafka consumer scaling method based on feedforward-feedback control

The invention belongs to the technical field of big data cloud computing, and discloses a Kafka consumer scaling method based on feedforward-feedback control, and the method comprises the following steps: reading the production rate of a producer as feedforward input information, and reading the mes...

Full description

Saved in:
Bibliographic Details
Main Authors CAO GUODONG, TAN GUOXING, XIAO LICHAO
Format Patent
LanguageChinese
English
Published 14.05.2024
Subjects
CALCULATING
COMPUTING
COUNTING
ELECTRIC DIGITAL DATA PROCESSING
PHYSICS
Online AccessGet full text

Cover

More Information
Summary:The invention belongs to the technical field of big data cloud computing, and discloses a Kafka consumer scaling method based on feedforward-feedback control, and the method comprises the following steps: reading the production rate of a producer as feedforward input information, and reading the message backlog Lag of a current consumer as feedback input information; analyzing output by combining the feedforward input information and the feedback input information to obtain the number of instances which should be set at the current moment; applying a constraint to the number of instances; adjusting the number of consumers into the number of instances meeting the constraint; by applying constraints and adjusting the number of consumers, when the production rate of a producer suddenly increases, instances can be synchronously increased without generating overstock, and at the same time, instances can be increased according to the overstock condition when the handling capacity of the consumers is reduced due to
Bibliography:Application Number: CN202311693745