Method for constructing depth information transmission model for multi-modal intensive prediction

• Main Authors: WU HEFENG, LIN JING, WANG QING, LIU LINGBO, ZHANG XIAOYU
• Format: Patent
• Language: Chinese; English
• Published: 23.02.2021
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; HANDLING RECORD CARRIERS; PHYSICS; PRESENTATION OF DATA; RECOGNITION OF DATA; RECORD CARRIERS

The invention provides a method for constructing a depth information transmission model for multi-modal intensive prediction, and the method comprises the steps: firstly constructing a plurality of sub-networks which are used for the representation learning of an RGB image or a heat map, and then constructing a sub-network which is used for mode sharing; then, constructing an information aggregation and distribution module IADM for completing translation-invariant information extraction, information aggregation transmission and information distribution transmission; by learning multi-modal alignment representation, a multi-modal intensive prediction framework containing an information aggregation distribution module is established, complementary information between different modals can befully captured, and information integration is well completed. In various multi-modal density prediction tasks, the method shows effectiveness and universality. 本发明提供一种多模态密集预测的深度信息传输模型的构建方法，该方法首先，构建多个子网络是用于RGB图或热图表征学习，再构建一个子网