Interaction-Based Active Perception Method and Vibration-Audio-Visual Information Fusion for Asteroid Surface Material Identification

• Published in: IEEE transactions on instrumentation and measurement Vol. 73; pp. 1 - 14
• Main Authors: Zhang, Jun, Xiao, Yi, Ding, Yizhuang, Chen, Liuchen, Song, Aiguo
• Format: Journal Article
• Language: English
• Published: New York The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
• Subjects: Asteroid missions; Audio data; Audio signals; Data integration; Image classification; Machine learning; Microgravity; Recognition; Robot sensors; Space exploration; Vibration; Vibration perception
• ISSN: 0018-9456; 1557-9662
• DOI: 10.1109/TIM.2024.3351245

Asteroid exploration is challenging due to its unknown surface material and microgravity. This article proposes an interaction-based active perception (IBAP) framework and a vibration-audio-visual information fusion (VAVIF) method for asteroid surface material identification. First, a robotic sensor node was designed to acquire vibration and audio signals when impacting the surface. With the help of the “interaction” between the node and surface material, a camera records visual images of the material splashing. Then, we proposed a SimpleCNN (SCNN) model to explore the rich features of the fusion images converted from the vibration signals for material identification. We also proposed a bidirectional RNN with attention mechanism (AB-RNN) model to fuse the high- and low-frequency vibration signals to improve the recognition performance. Results showed that SCNN and AB-RNN have better performance than the state-of-the-art models. In addition, state-of-the-art machine learning models are used to classify the visual images and audio signals for identification. Moreover, we utilized the output of multiple classifiers to construct a new dataset and employed ensemble learning (EL) for training and testing. The EL-based VAVIF obtained a recognition accuracy of 99.7%, higher than any individual learner. The results indicate that IBAP and VAVIF make material identification more accurate and robust, which can improve the success rate of an asteroid exploration mission.