3D Tooth Segmentation and Labeling Using Deep Convolutional Neural Networks

• Published in: IEEE transactions on visualization and computer graphics Vol. 25; no. 7; pp. 2336 - 2348
• Main Authors: Xu, Xiaojie, Liu, Chang, Zheng, Youyi
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.07.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: 3D mesh segmentation; Air bubbles; Algorithms; Artificial neural networks; Boundary-aware simplification; Clustering; deep convolutional neural networks; Dentistry; Dentition; Feature extraction; Finite element method; fuzzy clustering; Geometry; Humans; Image segmentation; Imaging, Three-Dimensional - methods; Labeling; Labelling; Neural networks; Neural Networks, Computer; Optimization; Radiography, Dental - methods; Segmentation; Solid modeling; Structural hierarchy; Teeth; Three dimensional models; Three-dimensional displays; Tomography, X-Ray Computed - methods; Tooth - diagnostic imaging
• ISSN: 1077-2626; 1941-0506; 1941-0506
• DOI: 10.1109/TVCG.2018.2839685

In this paper, we present a novel approach for 3D dental model segmentation via deep Convolutional Neural Networks (CNNs). Traditional geometry-based methods tend to receive undesirable results due to the complex appearance of human teeth (e.g., missing/rotten teeth, feature-less regions, crowding teeth, extra medical attachments, etc.). Furthermore, labeling of individual tooth is hardly enabled in traditional tooth segmentation methods. To address these issues, we propose to learn a generic and robust segmentation model by exploiting deep Neural Networks, namely NNs. The segmentation task is achieved by labeling each mesh face. We extract a set of geometry features as face feature representations. In the training step, the network is fed with those features, and produces a probability vector, of which each element indicates the probability a face belonging to the corresponding model part. To this end, we extensively experiment with various network structures, and eventually arrive at a 2-level hierarchical CNNs structure for tooth segmentation: one for teeth-gingiva labeling and the other for inter-teeth labeling. Further, we propose a novel boundary-aware tooth simplification method to significantly improve efficiency in the stage of feature extraction. After CNNs prediction, we do graph-based label optimization and further refine the boundary with an improved version of fuzzy clustering. The accuracy of our mesh labeling method exceeds that of the state-of-art geometry-based methods, reaching 99.06 percent measured by area which is directly applicable in orthodontic CAD systems. It is also robust to any possible foreign matters on model surface, e.g., air bubbles, dental accessories, and many more.