Tensor-based optimization method for memory management of a deep-learning GPU and system thereof

The present disclosure relates to a tensor-based optimization method for GPU memory management of deep learning, at least comprising steps of: executing at least one computing operation, which gets tensors as input and generates tensors as output; when one said computing operation is executed, track...

Full description

Saved in:
Bibliographic Details
Main Authors Ma, Weiliang, Xiong, Qian, Peng, Xuan, Jin, Hai, Dai, Hulin, Shi, Xuanhua
Format Patent
LanguageEnglish
Published 11.04.2023
Subjects
CALCULATING
COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
COMPUTING
COUNTING
ELECTRIC DIGITAL DATA PROCESSING
IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
PHYSICS
Online AccessGet full text

Cover

More Information
Summary:The present disclosure relates to a tensor-based optimization method for GPU memory management of deep learning, at least comprising steps of: executing at least one computing operation, which gets tensors as input and generates tensors as output; when one said computing operation is executed, tracking access information of the tensors, and setting up a memory management optimization decision based on the access information, during a first iteration of training, performing memory swapping operations passively between a CPU memory and a GPU memory so as to obtain the access information about the tensors regarding a complete iteration; according to the obtained access information about the tensors regarding the complete iteration, setting up a memory management optimization decision; and in a successive iteration, dynamically adjusting the set optimization decision of memory management according to operational feedbacks.
Bibliography:Application Number: US202016946690