SYSTEM AND METHOD FOR THROTTLING SERVICE REQUESTS HAVING NON-UNIFORM WORKLOADS

A system that provides services to clients may receive and service requests, various ones of which may require different amounts of work. The system may determine whether it is operating in an overloaded or underloaded state based on a current work throughput rate, a target work throughput rate, a m...

Full description

Saved in:
Bibliographic Details
Main Authors Lutz, David Alan, Rath, Timothy Andrew, Xiao, Wei, Filipe, Miguel Mascarenhas, Maccanti, Maximiliano, Yanacek, David Craig
Format Patent
LanguageEnglish
Published 07.09.2023
Subjects
CALCULATING
COMPUTING
COUNTING
ELECTRIC COMMUNICATION TECHNIQUE
ELECTRIC DIGITAL DATA PROCESSING
ELECTRICITY
PHYSICS
TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHICCOMMUNICATION
Online AccessGet full text

Cover

More Information
Summary:A system that provides services to clients may receive and service requests, various ones of which may require different amounts of work. The system may determine whether it is operating in an overloaded or underloaded state based on a current work throughput rate, a target work throughput rate, a maximum request rate, or an actual request rate, and may dynamically adjust the maximum request rate in response. For example, if the maximum request rate is being exceeded, the maximum request rate may be raised or lowered, dependent on the current work throughput rate. If the target or committed work throughput rate is being exceeded, but the maximum request rate is not being exceeded, a lower maximum request rate may be proposed. Adjustments to the maximum request rate may be made using multiple incremental adjustments. Service request tokens may be added to a leaky token bucket at the maximum request rate.
Bibliography:Application Number: US202318178379