Fully convolutional neural networks for LIDAR–camera fusion for pedestrian detection in autonomous vehicle

• Published in: Multimedia tools and applications Vol. 82; no. 16; pp. 25107 - 25130
• Main Authors: Alfred Daniel, J, Chandru Vignesh, C, Muthu, Bala Anand, Senthil Kumar, R, Sivaparthipan, CB, Marin, Carlos Enrique Montenegro
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.07.2023; Springer Nature B.V
• Subjects: Algorithms; Artificial neural networks; Automobiles; Autonomous cars; Autonomous vehicles; Cameras; Computer Communication Networks; Computer Science; Computer vision; Data Structures and Information Theory; Lidar; Multimedia Information Systems; Multisensor fusion; Neural networks; Parameter sensitivity; Pedestrians; Performance evaluation; Special Purpose and Application-Based Systems; Vehicle; Fusion for pedestrian; Convolutional neural networks; Detection in autonomous; LIDAR–camera
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-023-14417-x

Pedestrian detection appears to be an integral part of a vast array of vision-based technologies, ranging from item recognition and monitoring via surveillance cameras to, more recently, driverless cars or autonomous vehicles. Moreover, due to the rapid development of Convolutional Neural Networks (CNN) for object identification, pedestrian detection has reached a very high level of performance in dataset training and evaluation environment in autonomous vehicles. In order to attain object identification and pedestrian detection, a sensor fusion mechanism named Fully Convolutional Neural networks for LIDAR–camera fusion is proposed, which combines Lidar data with multiple camera images to provide an optimal solution for pedestrian detection. The system model proposes a separate algorithm for image fusion in pedestrian detection. Further, architecture and framework are designed for Fully Convolutional Neural networks for LIDAR–camera fusion for Pedestrian detection. In addition, a fully functional algorithm for Pedestrians detection and identification is proposed to precisely locate the pedestrian in the range of 10 to 30 m. Finally, the proposed model’s performance is evaluated based on multiple parameters such as Precision, Sensitivity, Accuracy, etc.; hence the proposed system model has proven to be effective comparatively.