RGB-T target tracking method based on multi-modal hierarchical relation modeling

• Main Authors: YAO RUI, ZHOU YONG, LIU BING, ZHU HANCHENG, SHAO ZHIWEN, ZHAO JIAQI, QIU JIAZHU
• Format: Patent
• Language: Chinese; English
• Published: 11.08.2023
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; PHYSICS

The invention discloses an RGB-T target tracking method based on multi-modal hierarchical relation modeling, and the method comprises the steps: carrying out the gradual gathering and fusion of multi-modal image features at a plurality of stages of image feature learning through a stacked multi-layer Transform encoder structure through a self-attention mechanism. In the multi-modal interaction process of the whole network, an image block-based dynamic component feature fusion module is utilized to dynamically solve the importance degree of visible light information of each area in a tracking scene, so that the interaction of the visible light information and infrared information in the tracking process is adjusted, the method better adapts to a complex scene, and the tracking efficiency is improved. And better tracking performance is obtained. 本发明公开了一种本发明公开了一种基于多模态层次关系建模的RGB-T目标跟踪方法，通过堆叠多层的Transformer编码器结构，利用自注意力机制在图像特征学习的多个阶段渐进式地聚集并融合多模态图像特征。在整个网络的多模态交互过程中，利用基于图像块的动态部件特征融合模块，动态求解跟踪场景中每个区域的可见光信息的重要程度，从而调节可见光信