Multi-mode cooperative sensing system and method for intelligent network connection vehicle

• Main Authors: TANG TAO, SHI SHANWEI, ZHANG CHAOKUN, HE SHEN
• Format: Patent
• Language: Chinese; English
• Published: 19.04.2024
• Subjects: CALCULATING; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS

The invention discloses a multi-modal cooperative sensing system and method for an intelligent networked vehicle, and the method comprises the steps: carrying out the single-modal fusion of LiDAR features and LiDAR features from different vehicles, and RGB image features and RGB image features through employing a self-attention mechanism after the multi-modal vehicle features are extracted and preprocessed; feature alignment is carried out on the LiDAR-BEV features and the RGB-BEV features; and stacking the RGB-BEV feature and the LiDAR-BEV feature which are aligned to generate a pre-fused BEV feature, taking the pre-fused BEV feature as an initialized query Query of a cross attention mechanism, constructing a Key and Value matrix by using the LiDAR-BEV feature after feature alignment, and obtaining a BEV fusion feature by using the cross attention mechanism. Compared with the prior art, the method has the advantages that a more comprehensive and reliable sensing solution is realized, the robustness and relia