Neural implicit shape modeling for small planetary bodies from multi-view images using a mask-based classification sampling strategy

• Published in: ISPRS journal of photogrammetry and remote sensing Vol. 212; pp. 122 - 145
• Main Authors: Chen, Hao, Hu, Xuanyu, Willner, Konrad, Ye, Zhen, Damme, Friedrich, Gläser, Philipp, Zheng, Yongjie, Tong, Xiaohua, Hußmann, Hauke, Oberst, Jürgen
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.06.2024
• Subjects: Mask-based classification sampling strategy; Multi-view images; Neural implicit method; Shape modeling; Small planetary bodies; Mask-based classification sampling strategy; Shape modeling; Neural implicit method; Small planetary bodies; Multi-view images
• ISSN: 0924-2716; 1872-8235
• DOI: 10.1016/j.isprsjprs.2024.04.029

Shape modeling is an indispensable task for spacecraft exploration of small planetary bodies. Traditional image-based techniques, such as stereo-photogrammetry or structure-from-motion + multi-view stereo, and stereo-photoclinometry, typically use a large number of images taken under favorable conditions for fine shape modeling, often requiring a long time for data acquisition and processing. Here, a novel neural implicit method, encoded by fully connected neural networks, is proposed for shape modeling using a sparse image set. The positions of surrounding points (SPs) with multi-scale receptive fields of a given input point are used as additional inputs for the network training, providing neighboring information. For fine-scale terrain features, a mask-based classification sampling strategy is developed to mitigate over-smoothing encountered by neural implicit methods. The effectiveness of our method is validated on two asteroids of distinct shapes, Itokawa and Ryugu, using 52 and 70 images, respectively. Comparative experiments demonstrate that the mask-based strategy, combined with the SPs configuration, accelerates network convergence for extracting fine surface details while minimizing the occurrence of artifacts. The proposed method can generate comprehensive shape models even in regions with restricted camera coverage, and the resulting models are consistent with those from traditional methods using larger image sets. Besides, the training process is executed in an end-to-end fashion, requiring limited manual intervention, and our method can readily be applied to other small planetary bodies. The source code and generated shape models in this study are available at https://doi.org/10.5281/zenodo.10909252.