Pillar Attention Encoder for Adaptive Cooperative Perception

• Published in: IEEE internet of things journal Vol. 11; no. 14; pp. 24998 - 25009
• Main Authors: Bai, Zhengwei, Wu, Guoyuan, Barth, Matthew J., Qiu, Hang, Liu, Yongkang, Sisbot, Emrah Akin, Oguchi, Kentaro
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 15.07.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: 3-D object detection; Adaptive systems; Automation; Bandwidth; Coders; Communication; connected and automated vehicles; cooperative perception (CP); Data mining; Feature extraction; feature filtering; Object recognition; Perception; Sensors; Shape; Three-dimensional displays; transformer
• ISSN: 2327-4662; 2327-4662
• DOI: 10.1109/JIOT.2024.3390552

Interest in cooperative perception (CP) is growing quickly due to its remarkable performance in improving perception capabilities for connected and automated vehicles. This improvement is crucial, especially for automated driving scenarios in which perception performance is one of the main bottlenecks to the development of safety and efficiency. However, current CP methods typically assume that all collaborating vehicles have enough communication bandwidth to share all features with an identical spatial size, which is impractical for real-world scenarios. In this article, we propose adaptive CP, a new CP framework that is not limited by the aforementioned assumptions, aiming to enable CP under more realistic and challenging conditions. To support this, a novel feature encoder is proposed and named pillar attention encoder. A pillar attention mechanism is designed to extract the feature data while considering its significance for the perception task. An adaptive feature filter is proposed to adjust the size of the feature data for sharing by considering the importance value of the feature. Experiments are conducted for cooperative object detection from multiple vehicle-based and infrastructure-based LiDAR sensors under various communication conditions. Results demonstrate that our method can successfully handle dynamic communication conditions and improve the mean average precision by 10.18% when compared with the state-of-the-art feature encoder.