Interruption-Aware Cooperative Perception for V2X Communication-Aided Autonomous Driving

• Published in: IEEE transactions on intelligent vehicles Vol. 9; no. 4; pp. 4698 - 4714
• Main Authors: Ren, Shunli, Lei, Zixing, Wang, Zi, Dianati, Mehrdad, Wang, Yafei, Chen, Siheng, Zhang, Wenjun
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.04.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: 3D object detection; autonomous driving; Autonomous vehicles; Communication; Cooperation; cooperative perception; Datasets; Distillation; Perception; Prediction models; Predictive models; Recovery; Safety; Sensors; Task analysis; Three-dimensional displays; Vehicle-to-everything
• ISSN: 2379-8858; 2379-8904
• DOI: 10.1109/TIV.2024.3371974

Cooperative perception can significantly improve the perception performance of autonomous vehicles beyond the limited perception ability of individual vehicles by exchanging information with neighbor agents through V2X communication. However, most existing work assume ideal communication among agents, ignoring the significant and common interruption issues caused by imperfect V2X communication, where cooperation agents can not receive cooperative messages successfully and thus fail to achieve cooperative perception, leading to safety risks. To fully reap the benefits of cooperative perception in practice, we propose V2X communication INterruption-aware COoperative Perception (V2X-INCOP), a cooperative perception system robust to communication interruption for V2X communication-aided autonomous driving, which leverages historical cooperation information to recover missing information due to the interruptions and alleviate the impact of the interruption issue. To achieve comprehensive recovery, we design a communication-adaptive multi-scale spatial-temporal prediction model to extract multi-scale spatial-temporal features based on V2X communication conditions and capture the most significant information for the prediction of the missing information. To further improve recovery performance, we adopt a knowledge distillation framework to give explicit and direct supervision to the prediction model and a curriculum learning strategy to stabilize the training of the model. Experiments on three public cooperative perception datasets demonstrate that the proposed method is effective in alleviating the impacts of communication interruption on cooperative perception. V2X-INCOP outperforms state-of-the-art cooperative perception methods and has a cooperative perception gain up to 14.06%, 13.9%, and 12.07% over individual perception on average of different packet drop rates on OPV2V, V2X-Sim, and Dair-V2X datasets, respectively.