Ditto: Building Digital Twins of Articulated Objects from Interaction

Digitizing physical objects into the virtual world has the potential to unlock new research and applications in embodied AI and mixed reality. This work focuses on recreating interactive digital twins of real-world articulated objects, which can be directly imported into virtual environments. We int...

Full description

Saved in:
Bibliographic Details
Published inProceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 5606 - 5616
Main Authors Jiang, Zhenyu, Hsu, Cheng-Chun, Zhu, Yuke
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2022
Subjects
3D from multi-view and sensors; Physics-based vision and shape-from-X; Representation learning
Buildings
Estimation
Geometry
Solid modeling
Three-dimensional displays
Virtual environments
Visualization
Online AccessGet full text

Cover

More Information
Summary:Digitizing physical objects into the virtual world has the potential to unlock new research and applications in embodied AI and mixed reality. This work focuses on recreating interactive digital twins of real-world articulated objects, which can be directly imported into virtual environments. We introduce Ditto to learn articulation model estimation and 3D geometry reconstruction of an articulated object through interactive perception. Given a pair of visual observations of an articulated object before and after interaction, Ditto reconstructs part-level geometry and estimates the articulation model of the object. We employ implicit neural representations for joint geometry and articulation modeling. Our experiments show that Ditto effectively builds digital twins of articulated objects in a category-agnostic way. We also apply Ditto to real-world objects and deploy the recreated digital twins in physical simulation. Code and additional results are available at https://ut-austin-rpl.github.io/Ditto/
ISSN:1063-6919
DOI:10.1109/CVPR52688.2022.00553