Operator concurrency adjusting and optimizing method and device for stream processing system
The invention provides an operator concurrency adjusting and optimizing method and device for a stream processing system, and the method comprises the steps: carrying out the excessive capacity expansion adjusting and optimizing of a stream processing task in a backpressure state, so as to enable ea...
Saved in:
Main Authors | , , , |
---|---|
Format | Patent |
Language | Chinese English |
Published |
06.06.2023
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | The invention provides an operator concurrency adjusting and optimizing method and device for a stream processing system, and the method comprises the steps: carrying out the excessive capacity expansion adjusting and optimizing of a stream processing task in a backpressure state, so as to enable each stream processing task to be in a to-be-reduced state at present, and to serve as a target stream processing task; training an abstract operator in the stream processing system according to historical tuning information to generate a GP model, and calculating to obtain a workload adaptability score and a self-design WOP acquisition function according to a historical estimated load in the GP model and a current workload of the target stream processing task; and determining the optimal concurrency of the abstract operator according to the GP model and the self-design WOP acquisition function. Compared with the prior art, the method does not depend on an adjustment and optimization built-in rule, makes full use of |
---|---|
Bibliography: | Application Number: CN202310068747 |