FlyBy CNN: A 3D surface segmentation framework

• Published in: Proceedings of SPIE, the international society for optical engineering Vol. 11596
• Main Authors: Boumbolo, Louis, Dumont, Maxime, Brosset, Serge, Bianchi, Jonas, Ruellas, Antonio, Gurgel, Marcela, Massaro, Camila, Del Castillo, Aron Aliaga, Ioshida, Marcos, Yatabe, Marilia, Benavides, E, Rios, Hector, Soki, Fabiana, Neiva, Gisele, Paniagua, Beatriz, Cevidanes, Lucia, Styner, Martin, Prieto, Juan C
• Format: Journal Article
• Language: English
• Published: United States 01.02.2021
• Subjects: segmentation; shape analysis; intra oral surface; mesh
• ISSN: 0277-786X
• DOI: 10.1117/12.2582205

In this paper, we present FlyBy CNN, a novel deep learning based approach for 3D shape segmentation. FlyByCNN consists of sampling the surface of the 3D object from different view points and extracting surface features such as the normal vectors. The generated 2D images are then analyzed via 2D convolutional neural networks such as RUNETs. We test our framework in a dental application for segmentation of intra-oral surfaces. The RUNET is trained for the segmentation task using image pairs of surface features and image labels as ground truth. The resulting labels from each segmented image are put back into the surface thanks to our sampling approach that generates 1-1 correspondence of image pixels and triangles in the surface model. The segmentation task achieved an accuracy of 0.9.