DINA: Deformable INteraction Analogy

• Published in: Graphical models Vol. 133; p. 101217
• Main Authors: Huang, Zeyu, Dai, Sisi, Xu, Kai, Zhang, Hao, Huang, Hui, Hu, Ruizhen
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.06.2024; Elsevier
• Subjects: Imitation learning; Interaction modeling; Optimization; Shape analysis; Shape analysis; Interaction modeling; Imitation learning; Optimization
• ISSN: 1524-0703; 1524-0711
• DOI: 10.1016/j.gmod.2024.101217

We introduce deformable interaction analogy (DINA) as a means to generate close interactions between two 3D objects. Given a single demo interaction between an anchor object (e.g. a hand) and a source object (e.g. a mug grasped by the hand), our goal is to generate many analogous 3D interactions between the same anchor object and various new target objects (e.g. a toy airplane), where the anchor object is allowed to be rigid or deformable. To this end, we optimize the pose or shape of the anchor object to adapt it to a new target object to mimic the demo. To facilitate the optimization, we advocate using interaction interface (ITF), defined by a set of points sampled on the anchor object, as a descriptive and robust interaction representation that is amenable to non-rigid deformation. We model similarity between interactions using ITF, while for interaction analogy, we transform the ITF, either rigidly or non-rigidly, to guide the feature matching to the reposing and deformation of the anchor object. Qualitative and quantitative experiments show that our ITF-guided deformable interaction analogy works surprisingly well even with simple distance features compared to variants of state-of-the-art methods that utilize more sophisticated interaction representations and feature learning from large datasets. [Display omitted] •We introduce deformable interaction analogy (DINA) as a means to generate interactions between two 3D objects.•We propose interaction interface (ITF) which is a descriptive and robust interaction representation.•We conduct extensive experiments to show the superiority of our approach on hand grasping generation task.