Predicting Performance of Heterogeneous AI Systems with Discrete-Event Simulations
In recent years, artificial intelligence (AI) technologies have found industrial applications in various fields. AI systems typically possess complex software and heterogeneous CPU/GPU hardware architecture, making it difficult to answer basic questions considering performance evaluation and softwar...
Saved in:
Main Authors | , , |
---|---|
Format | Journal Article |
Language | English |
Published |
07.04.2022
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | In recent years, artificial intelligence (AI) technologies have found
industrial applications in various fields. AI systems typically possess complex
software and heterogeneous CPU/GPU hardware architecture, making it difficult
to answer basic questions considering performance evaluation and software
optimization. Where is the bottleneck impeding the system? How does the
performance scale with the workload? How the speed-up of a specific module
would contribute to the whole system? Finding the answers to these questions
through experiments on the real system could require a lot of computational,
human, financial, and time resources. A solution to cut these costs is to use a
fast and accurate simulation model preparatory to implementing anything in the
real system. In this paper, we propose a discrete-event simulation model of a
high-load heterogeneous AI system in the context of video analytics. Using the
proposed model, we estimate: 1) the performance scalability with the increasing
number of cameras; 2) the performance impact of integrating a new module; 3)
the performance gain from optimizing a single module. We show that the
performance estimation accuracy of the proposed model is higher than 90%. We
also demonstrate, that the considered system possesses a counter-intuitive
relationship between workload and performance, which nevertheless is correctly
inferred by the proposed simulation model. |
---|---|
DOI: | 10.48550/arxiv.2204.03332 |