Tuning large data infrastructures

An automated tuning service is used to automatically tune, or modify, the operational parameters of a large-scale cloud infrastructure. The tuning service performs automated and fully data/model-driven configuration based from learning various real-time performance of the cloud infrastructure. Such...

Full description

Saved in:
Bibliographic Details
Main Authors Zhu, Yiwen, Krishnan, Subramaniam Venkatraman, Karanasos, Konstantinos, Curino, Carlo, Darbha, Sudhir, Tarte, Isha
Format Patent
LanguageEnglish
Published 23.01.2024
Subjects
CALCULATING
COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
COMPUTING
COUNTING
ELECTRIC DIGITAL DATA PROCESSING
PHYSICS
Online AccessGet full text

Cover

More Information
Summary:An automated tuning service is used to automatically tune, or modify, the operational parameters of a large-scale cloud infrastructure. The tuning service performs automated and fully data/model-driven configuration based from learning various real-time performance of the cloud infrastructure. Such performance is identified through monitoring various telemetric data of the cloud infrastructure. The tuning service leverages a mix of domain knowledge and principled data-science to capture the essence of our cluster dynamic behavior in a collection of descriptive machine learning (ML) models. The ML models power automated optimization procedures for parameter tuning, and inform administrators in most tactical and strategical engineering/capacity decisions (such as hardware and datacenter design, software investments, etc.). Rich "observational" models (models collected without modifying the system) are combined with judicious use of "fighting" (testing in production), allowing the tuning service to automatically configure operational parameters of a large cloud infrastructure for a broad range of applications.
Bibliography:Application Number: US202117221755