On Color-, Infrared-, and Multimodal-Stereo Approaches to Pedestrian Detection

• Published in: IEEE transactions on intelligent transportation systems Vol. 8; no. 4; pp. 619 - 629
• Main Authors: Krotosky, S.J., Trivedi, M.M.
• Format: Journal Article
• Language: English
• Published: Piscataway, NJ IEEE 01.12.2007; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Active safety; Applied sciences; Artificial intelligence; Cameras; Classification; collision avoidance; Color; Computer science; control theory; systems; Control theory. Systems; Design engineering; Exact sciences and technology; Ground, air and sea transportation, marine construction; Image color analysis; Infrared; Infrared cameras; Infrared detectors; Infrared imaging; intelligent vehicles; Layout; Obstacles; Pattern recognition. Digital image processing. Computational geometry; Pedestrians; person detection; Robotics; Robustness; Testing; tracking; Vehicle detection; Vehicle safety; Vehicles; collision avoidance; Active safety; person detection; tracking; intelligent vehicles; Infrared thermography; Tracking; Infrared camera; Permutation; Pedestrian; Multiple view; Image analysis; Active system; Disparity; Image sensor; Collision avoidance
• ISSN: 1524-9050; 1558-0016
• DOI: 10.1109/TITS.2007.908722

This paper presents an analysis of color-, infrared-, and multimodal-stereo approaches to pedestrian detection. We design a four-camera experimental testbed consisting of two color and two infrared cameras for capturing and analyzing various configuration permutations for pedestrian detection. We incorporate this four-camera system in a test vehicle and conduct comparative experiments of stereo-based approaches to obstacle detection using unimodal color and infrared imageries. A detailed analysis of the color and infrared features used to classify detected obstacles into pedestrian regions is used to motivate the development of a multimodal solution to pedestrian detection. We propose a multimodal trifocal framework consisting of a stereo pair of color cameras coupled with an infrared camera. We use this framework to combine multimodal-image features for pedestrian detection and to demonstrate that the detection performance is significantly higher when color, disparity, and infrared features are used together. This result motivates experiments and discussion toward achieving multimodal-feature combination using a single color and a single infrared camera arranged in a cross-spectral stereo pair. We demonstrate an approach to registering multiple objects across modalities and provide an experimental analysis that highlights issues and challenges of pursuing the cross-spectral approach to multimodal and multiperspective pedestrian analysis.