Satellite video multi-vehicle online tracking method and device based on motion tracking Transformer

• Main Authors: ZHANG SAI, XIE ZHULI, JIA YUTONG, WANG ZHENYU, LIU WEI, LI GONG, YIN YUNXIA, WANG GUOPING, LIU JIA, LI CHU, SU ZHIJUAN, PENG SIQING, WAN GANG, WU YITIAN, CONG DIANWEI, DENG PANKE
• Format: Patent
• Language: Chinese; English
• Published: 28.11.2023
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; PHYSICS

The invention discloses a satellite video multi-vehicle online tracking method and device based on a motion tracking Transform. The method comprises the following steps: acquiring a remote sensing video sequence containing a target; extracting feature information of the remote sensing video sequence by using a CNN backbone network; a self-attention module of a Transform encoder is used to encode the extracted feature information; at a decoder, respectively inputting three previous tracking query results into three sequential attention modules of the decoder so as to learn tracks of each target, and inputting outputs of the three sequential attention modules and encoded features into another attention module of the decoder; and mapping the output of the decoder through two output heads based on the FFN, and generating a tracking query result of the target. According to the tracking method for predicting the position of the shielded target by using the short-term memory information, in the tracking process, the