Tensor-based deep learning GPU memory management optimization method and system

• Main Authors: DAI HULIN, JIN HAI, PENG XUAN, MA WEILIANG, XIONG QIAN, SHI XUANHUA
• Format: Patent
• Language: Chinese; English
• Published: 28.04.2020
• Subjects: CALCULATING; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS

The invention relates to a tensor-based deep learning GPU memory management optimization method, which at least comprises the following steps: executing at least one calculation operation which takesa tensor as an input and generates a tensor as an output; when one computing operation is executed, tracking access information of the tensor, making a memory management optimization decision based onthe access information, passively performing memory exchange between a CPU memory and a GPU memory in the first iteration of training to obtain access information of the tensor of one complete iteration, and making a memory management optimization decision according to the obtained access information about the tensor of the complete iteration; and in subsequent iteration, dynamically adjusting the formulated memory management optimization decision according to feedback during operation. 本发明涉及一种基于张量的深度学习GPU内存管理优化方法，至少包括如下步骤：执行至少一个计算操作，其中，所述计算操作以张量为输入并产生张量作为输出；在一个计算操作被执行时，对张量的访问信息进行跟踪，并基于所述访问信息制定内存管理优化决策，在训练的第一次迭代中被动地在C