Tensor-based optimization method for memory management of a deep-learning GPU and system thereof
The present disclosure relates to a tensor-based optimization method for GPU memory management of deep learning, at least comprising steps of: executing at least one computing operation, which gets tensors as input and generates tensors as output; when one said computing operation is executed, track...
Saved in:
Main Authors | , , , , , |
---|---|
Format | Patent |
Language | English |
Published |
11.04.2023
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | The present disclosure relates to a tensor-based optimization method for GPU memory management of deep learning, at least comprising steps of: executing at least one computing operation, which gets tensors as input and generates tensors as output; when one said computing operation is executed, tracking access information of the tensors, and setting up a memory management optimization decision based on the access information, during a first iteration of training, performing memory swapping operations passively between a CPU memory and a GPU memory so as to obtain the access information about the tensors regarding a complete iteration; according to the obtained access information about the tensors regarding the complete iteration, setting up a memory management optimization decision; and in a successive iteration, dynamically adjusting the set optimization decision of memory management according to operational feedbacks. |
---|---|
Bibliography: | Application Number: US202016946690 |