Machines Learning for Mixed Reality The Milan Cathedral from Survey to Holograms

• Published in: Pattern Recognition. ICPR International Workshops and Challenges pp. 613 - 627
• Main Authors: Teruggi, Simone, Fassi, Francesco
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Classification; Cultural heritage; HoloLens2; Machine Learning; Mixed reality; Point cloud
• ISBN: 9783030687953; 3030687953
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-68796-0_44

In recent years, a complete 3D mapping of the Cultural Heritage (CH) has become fundamental before every other action could follow. Different survey techniques outputs could be combined in a 3D point cloud, completely describing the geometry of even the most complex object. These data very rich in metric quality can be used to extract 2D technical elaborations and advanced 3D representations to support conservation interventions and maintenance planning. The case of Milan Cathedral is outstanding. In the last 12 years, a multi-technique omni-comprehensive survey has been carried out to extract the technical representations that are used by the Veneranda Fabbrica (VF) del Duomo di Milano to plan its maintenance and conservation activities. Nevertheless, point cloud data lack structured information such as semantics and hierarchy among parts, fundamentals for 3D model interaction and database (DB) retrieval. In this context, the introduction of point cloud classification methods could improve data usage, model definition and analysis. In this paper, a Multi-level Multi-resolution (MLMR) classification approach is presented and tested on the large dataset of Milan Cathedral. The 3D point model, so structured, for the first time, is used directly in a Mixed Reality (MR) environment to develop an application that could benefit professional works, allowing to use 3D survey data on-site, supporting VF activities.