Generative 3D Part Assembly via Part-Whole-Hierarchy Message Passing
Generative 3D part assembly involves understanding part relationships and predicting their 6-DoF poses for assembling a realistic 3D shape. Prior work often focus on the geometry of individual parts, neglecting part-whole hierarchies of objects. Leveraging two key observations: 1) super-part poses p...
Saved in:
Main Authors | , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
27.02.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Generative 3D part assembly involves understanding part relationships and
predicting their 6-DoF poses for assembling a realistic 3D shape. Prior work
often focus on the geometry of individual parts, neglecting part-whole
hierarchies of objects. Leveraging two key observations: 1) super-part poses
provide strong hints about part poses, and 2) predicting super-part poses is
easier due to fewer superparts, we propose a part-whole-hierarchy message
passing network for efficient 3D part assembly. We first introduce super-parts
by grouping geometrically similar parts without any semantic labels. Then we
employ a part-whole hierarchical encoder, wherein a super-part encoder predicts
latent super-part poses based on input parts. Subsequently, we transform the
point cloud using the latent poses, feeding it to the part encoder for
aggregating super-part information and reasoning about part relationships to
predict all part poses. In training, only ground-truth part poses are required.
During inference, the predicted latent poses of super-parts enhance
interpretability. Experimental results on the PartNet dataset show that our
method achieves state-of-the-art performance in part and connectivity accuracy
and enables an interpretable hierarchical part assembly. Code is available at
https://github.com/pkudba/3DHPA. |
---|---|
AbstractList | Generative 3D part assembly involves understanding part relationships and
predicting their 6-DoF poses for assembling a realistic 3D shape. Prior work
often focus on the geometry of individual parts, neglecting part-whole
hierarchies of objects. Leveraging two key observations: 1) super-part poses
provide strong hints about part poses, and 2) predicting super-part poses is
easier due to fewer superparts, we propose a part-whole-hierarchy message
passing network for efficient 3D part assembly. We first introduce super-parts
by grouping geometrically similar parts without any semantic labels. Then we
employ a part-whole hierarchical encoder, wherein a super-part encoder predicts
latent super-part poses based on input parts. Subsequently, we transform the
point cloud using the latent poses, feeding it to the part encoder for
aggregating super-part information and reasoning about part relationships to
predict all part poses. In training, only ground-truth part poses are required.
During inference, the predicted latent poses of super-parts enhance
interpretability. Experimental results on the PartNet dataset show that our
method achieves state-of-the-art performance in part and connectivity accuracy
and enables an interpretable hierarchical part assembly. Code is available at
https://github.com/pkudba/3DHPA. |
Author | Hu, Wei Gao, Xiang Du, Bi'an Liao, Renjie |
Author_xml | – sequence: 1 givenname: Bi'an surname: Du fullname: Du, Bi'an – sequence: 2 givenname: Xiang surname: Gao fullname: Gao, Xiang – sequence: 3 givenname: Wei surname: Hu fullname: Hu, Wei – sequence: 4 givenname: Renjie surname: Liao fullname: Liao, Renjie |
BackLink | https://doi.org/10.48550/arXiv.2402.17464$$DView paper in arXiv |
BookMark | eNotj81KAzEUhbPQhVYfwJV5gYz5n3FZWm2Fii4KLoeb5KYNTKeSlMF5e-vY1YFzPg58t-SqP_ZIyIPglW6M4U-Qf9JQSc1lJWpt9Q1ZrrDHDKc0IFVL-gn5ROel4MF1Ix0STA372h87ZOt0JrPfj_QdS4EdnsdSUr-7I9cRuoL3l5yR7evLdrFmm4_V22K-YWBrzQL4RloAHzFai7HWKCwoFIob4xoZgkETRANaROeFDdp4K10dIlrJn52akcf_20mj_c7pAHls_3TaSUf9Ar7KR0I |
ContentType | Journal Article |
Copyright | http://arxiv.org/licenses/nonexclusive-distrib/1.0 |
Copyright_xml | – notice: http://arxiv.org/licenses/nonexclusive-distrib/1.0 |
DBID | AKY GOX |
DOI | 10.48550/arxiv.2402.17464 |
DatabaseName | arXiv Computer Science arXiv.org |
DatabaseTitleList | |
Database_xml | – sequence: 1 dbid: GOX name: arXiv.org url: http://arxiv.org/find sourceTypes: Open Access Repository |
DeliveryMethod | fulltext_linktorsrc |
ExternalDocumentID | 2402_17464 |
GroupedDBID | AKY GOX |
ID | FETCH-LOGICAL-a674-dac826aacfef66ef74e16a3e13055b82dd5e5d18a41fbc16d45c62b7dfe6209b3 |
IEDL.DBID | GOX |
IngestDate | Fri Mar 29 12:25:22 EDT 2024 |
IsDoiOpenAccess | true |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-a674-dac826aacfef66ef74e16a3e13055b82dd5e5d18a41fbc16d45c62b7dfe6209b3 |
OpenAccessLink | https://arxiv.org/abs/2402.17464 |
ParticipantIDs | arxiv_primary_2402_17464 |
PublicationCentury | 2000 |
PublicationDate | 2024-02-27 |
PublicationDateYYYYMMDD | 2024-02-27 |
PublicationDate_xml | – month: 02 year: 2024 text: 2024-02-27 day: 27 |
PublicationDecade | 2020 |
PublicationYear | 2024 |
Score | 1.9169219 |
SecondaryResourceType | preprint |
Snippet | Generative 3D part assembly involves understanding part relationships and
predicting their 6-DoF poses for assembling a realistic 3D shape. Prior work
often... |
SourceID | arxiv |
SourceType | Open Access Repository |
SubjectTerms | Computer Science - Computer Vision and Pattern Recognition |
Title | Generative 3D Part Assembly via Part-Whole-Hierarchy Message Passing |
URI | https://arxiv.org/abs/2402.17464 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwdV3NS0MxDC_bTl5EUZmf9OC1uPalfa_H4ZxD8OMwcbfRvqY6UJFtDvff2_RN9OI1SQ9NoEma5BfGziX45NidFQGhFADYE1VQTlgXaoO6oKE96ra4M6NHuJnoSYvxn1kYN_-arRp8YL-4oK9_arkx0GZtpahl6_p-0hQnMxTXRv5XLsWYmfTHSQx32PYmuuP9xhy7rIXve2zQQDvTu8KLAX9IxuJUa33zr2u-mrlMEU-0qFaMZjQQXL-s-S3tJnnGxFxQOr_PxsOr8eVIbJYXCGdKEMHVKXB3ro4YjcFYAkrjCpSEsOUrFYJGHWTlQEZfSxNA10b5MkQ0qmd9ccA6Kf_HLuPSgo5OW7QKoReoMunTURsBg6kKf8i6-crTjwafYkramGZtHP3POmZbKvnnPJ1dnrDOcv6Jp8m_Lv1ZVvI3YmZ6zQ |
link.rule.ids | 228,230,786,891 |
linkProvider | Cornell University |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Generative+3D+Part+Assembly+via+Part-Whole-Hierarchy+Message+Passing&rft.au=Du%2C+Bi%27an&rft.au=Gao%2C+Xiang&rft.au=Hu%2C+Wei&rft.au=Liao%2C+Renjie&rft.date=2024-02-27&rft_id=info:doi/10.48550%2Farxiv.2402.17464&rft.externalDocID=2402_17464 |