Identifying Style of 3D Shapes using Deep Metric Learning

• Published in: Computer graphics forum Vol. 35; no. 5; pp. 207 - 215
• Main Authors: Lim, Isaak, Gehre, Anne, Kobbelt, Leif
• Format: Journal Article
• Language: English
• Published: Oxford Blackwell Publishing Ltd 01.08.2016
• Subjects: Analysis; Categories and Subject Descriptors (according to ACM CCS); Classification; Collection; Computer graphics; deep metric learning; I.3.5 [Computer Graphics]: Computational Geometry and Object Modeling; Image processing systems; Learning; Neural networks; Similarity; State of the art; Studies; style similarity; Training
• ISSN: 0167-7055; 1467-8659
• DOI: 10.1111/cgf.12977

We present a method that expands on previous work in learning human perceived style similarity across objects with different structures and functionalities. Unlike previous approaches that tackle this problem with the help of hand‐crafted geometric descriptors, we make use of recent advances in metric learning with neural networks (deep metric learning). This allows us to train the similarity metric on a shape collection directly, since any low‐ or high‐level features needed to discriminate between different styles are identified by the neural network automatically. Furthermore, we avoid the issue of finding and comparing sub‐elements of the shapes. We represent the shapes as rendered images and show how image tuples can be selected, generated and used efficiently for deep metric learning. We also tackle the problem of training our neural networks on relatively small datasets and show that we achieve style classification accuracy competitive with the state of the art. Finally, to reduce annotation effort we propose a method to incorporate heterogeneous data sources by adding annotated photos found online in order to expand or supplant parts of our training data.