MC-NeRF: Multi-Camera Neural Radiance Fields for Multi-Camera Image Acquisition Systems
Neural Radiance Fields (NeRF) use multi-view images for 3D scene representation, demonstrating remarkable performance. As one of the primary sources of multi-view images, multi-camera systems encounter challenges such as varying intrinsic parameters and frequent pose changes. Most previous NeRF-base...
Saved in:
Published in | arXiv.org |
---|---|
Main Authors | , , , , , |
Format | Paper |
Language | English |
Published |
Ithaca
Cornell University Library, arXiv.org
22.03.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Neural Radiance Fields (NeRF) use multi-view images for 3D scene representation, demonstrating remarkable performance. As one of the primary sources of multi-view images, multi-camera systems encounter challenges such as varying intrinsic parameters and frequent pose changes. Most previous NeRF-based methods assume a unique camera and rarely consider multi-camera scenarios. Besides, some NeRF methods that can optimize intrinsic and extrinsic parameters still remain susceptible to suboptimal solutions when these parameters are poor initialized. In this paper, we propose MC-NeRF, a method that enables joint optimization of both intrinsic and extrinsic parameters alongside NeRF. The method also supports each image corresponding to independent camera parameters. First, we tackle coupling issue and the degenerate case that arise from the joint optimization between intrinsic and extrinsic parameters. Second, based on the proposed solutions, we introduce an efficient calibration image acquisition scheme for multi-camera systems, including the design of calibration object. Finally, we present an end-to-end network with training sequence that enables the estimation of intrinsic and extrinsic parameters, along with the rendering network. Furthermore, recognizing that most existing datasets are designed for a unique camera, we construct a real multi-camera image acquisition system and create a corresponding new dataset, which includes both simulated data and real-world captured images. Experiments confirm the effectiveness of our method when each image corresponds to different camera parameters. Specifically, we use multi-cameras, each with different intrinsic and extrinsic parameters in real-world system, to achieve 3D scene representation without providing initial poses. |
---|---|
AbstractList | Neural Radiance Fields (NeRF) use multi-view images for 3D scene representation, demonstrating remarkable performance. As one of the primary sources of multi-view images, multi-camera systems encounter challenges such as varying intrinsic parameters and frequent pose changes. Most previous NeRF-based methods assume a unique camera and rarely consider multi-camera scenarios. Besides, some NeRF methods that can optimize intrinsic and extrinsic parameters still remain susceptible to suboptimal solutions when these parameters are poor initialized. In this paper, we propose MC-NeRF, a method that enables joint optimization of both intrinsic and extrinsic parameters alongside NeRF. The method also supports each image corresponding to independent camera parameters. First, we tackle coupling issue and the degenerate case that arise from the joint optimization between intrinsic and extrinsic parameters. Second, based on the proposed solutions, we introduce an efficient calibration image acquisition scheme for multi-camera systems, including the design of calibration object. Finally, we present an end-to-end network with training sequence that enables the estimation of intrinsic and extrinsic parameters, along with the rendering network. Furthermore, recognizing that most existing datasets are designed for a unique camera, we construct a real multi-camera image acquisition system and create a corresponding new dataset, which includes both simulated data and real-world captured images. Experiments confirm the effectiveness of our method when each image corresponds to different camera parameters. Specifically, we use multi-cameras, each with different intrinsic and extrinsic parameters in real-world system, to achieve 3D scene representation without providing initial poses. |
Author | Fu, Mengyin Gao, Yu Liang, Hao Su, Lutong Yue, Yufeng Yang, Yi |
Author_xml | – sequence: 1 givenname: Yu surname: Gao fullname: Gao, Yu – sequence: 2 givenname: Lutong surname: Su fullname: Su, Lutong – sequence: 3 givenname: Hao surname: Liang fullname: Liang, Hao – sequence: 4 givenname: Yufeng surname: Yue fullname: Yue, Yufeng – sequence: 5 givenname: Yi surname: Yang fullname: Yang, Yi – sequence: 6 givenname: Mengyin surname: Fu fullname: Fu, Mengyin |
BookMark | eNqNyrsOgjAYQOHGaCIq79DEuQm0UombIRIdYEATR9LAjymBVnoZfHsdXNycznC-FZorrWCGAspYTNIdpUsUWttHUUT5niYJC9C9yEgJVX7AhR-cJJkYwQhcgjdiwJVopVAN4FzC0FrcafPrLqN4AD42k5dWOqkVvr6sg9Fu0KITg4Xw2zXa5qdbdiZPoycP1tW99kZ9Vk1TnsSMc87Yf-oNye9B2g |
ContentType | Paper |
Copyright | 2024. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
Copyright_xml | – notice: 2024. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
DBID | 8FE 8FG ABJCF ABUWG AFKRA AZQEC BENPR BGLVJ CCPQU DWQXO HCIFZ L6V M7S PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
DatabaseName | ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central (Alumni Edition) ProQuest Central UK/Ireland ProQuest Central Essentials ProQuest Central Technology Collection ProQuest One Community College ProQuest Central SciTech Premium Collection ProQuest Engineering Collection Engineering Database ProQuest - Publicly Available Content Database ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection |
DatabaseTitle | Publicly Available Content Database Engineering Database Technology Collection ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Technology Collection ProQuest SciTech Collection ProQuest Central China ProQuest Central ProQuest Engineering Collection ProQuest One Academic UKI Edition ProQuest Central Korea Materials Science & Engineering Collection ProQuest One Academic Engineering Collection |
DatabaseTitleList | Publicly Available Content Database |
Database_xml | – sequence: 1 dbid: 8FG name: ProQuest Technology Collection url: https://search.proquest.com/technologycollection1 sourceTypes: Aggregation Database |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Physics |
EISSN | 2331-8422 |
Genre | Working Paper/Pre-Print |
GroupedDBID | 8FE 8FG ABJCF ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS AZQEC BENPR BGLVJ CCPQU DWQXO FRJ HCIFZ L6V M7S M~E PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
ID | FETCH-proquest_journals_28651366633 |
IEDL.DBID | 8FG |
IngestDate | Thu Oct 10 16:38:32 EDT 2024 |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-proquest_journals_28651366633 |
OpenAccessLink | https://www.proquest.com/docview/2865136663?pq-origsite=%requestingapplication% |
PQID | 2865136663 |
PQPubID | 2050157 |
ParticipantIDs | proquest_journals_2865136663 |
PublicationCentury | 2000 |
PublicationDate | 20240322 |
PublicationDateYYYYMMDD | 2024-03-22 |
PublicationDate_xml | – month: 03 year: 2024 text: 20240322 day: 22 |
PublicationDecade | 2020 |
PublicationPlace | Ithaca |
PublicationPlace_xml | – name: Ithaca |
PublicationTitle | arXiv.org |
PublicationYear | 2024 |
Publisher | Cornell University Library, arXiv.org |
Publisher_xml | – name: Cornell University Library, arXiv.org |
SSID | ssj0002672553 |
Score | 3.5369146 |
SecondaryResourceType | preprint |
Snippet | Neural Radiance Fields (NeRF) use multi-view images for 3D scene representation, demonstrating remarkable performance. As one of the primary sources of... |
SourceID | proquest |
SourceType | Aggregation Database |
SubjectTerms | Calibration Cameras Datasets Image acquisition Optimization Parameters Radiance Representations System effectiveness Uniqueness |
Title | MC-NeRF: Multi-Camera Neural Radiance Fields for Multi-Camera Image Acquisition Systems |
URI | https://www.proquest.com/docview/2865136663 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1LSwMxEB60i-DNJz5qCeg16Cb79FJ02bUKu5RFsbeS7CYgWG277dXf3kzcKiL0GBJCEsLM5Ms33wBcMR4KLplPNYs09RBoQjVcqmMVcxX4tVSY75wXweDFexr5oxZwa1pa5domWkNdf1aIkV9jBqXLTbDN-9MZxapR-LvaltDYBsdFJTzMFM8efjAWFoQmYub_zKz1HdkeOEMxVfN92FIfB7BjKZdVcwiveUILVWa3xCbB0kQgPkRQLUO8k9JqBlSKZEgxa4iJLf-Oe5wYS0Duqtny7Zt2RVrx8SO4zNLnZEDXqxm396UZ_-6OH0PHPPzVCZDQq_34RlRuLEPPk7UIzGtNxVFYR1pKrU-hu2mms83d57DLjINGPhVjXegs5kt1YRzsQvbsKfbAuU-LYWla-Ve6AtY9hZg |
link.rule.ids | 786,790,12792,21416,33406,33777,43633,43838 |
linkProvider | ProQuest |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1bS8MwFD5oi-ibV7xMDehr0CW9-iJaVjpdyygT91aSNoGBl23d_r85tVNE2HNCSEI4ly_f-Q7ANeO-4JK5VLNAUweBJlTDpTpUIVeeW0mF9c5p5iUvztPYHbeAW93SKlc2sTHU1WeJGPkNVlB2uQm2-f10RrFrFP6uti00NsFGyc3AAvuxlw3zH5SFeb6Jmfk_Q9t4j3gX7KGYqvkebKiPfdhqSJdlfQCvaUQzlcd3pCmDpZFAhIigXoZ4I3mjGlAqEiPJrCYmuvw7r_9ubAF5KGfLyTfxirTy44dwFfdGUUJXuynaF1MXv-fjR2CZ1F8dA_Gdyg1vRdkNpe84shKeyddUGPhVoKXU-gQ661Y6XT98CdvJKB0Ug372fAY7zLhrZFcx1gFrMV-qc-NuF_KivdMvLieHJA |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=MC-NeRF%3A+Multi-Camera+Neural+Radiance+Fields+for+Multi-Camera+Image+Acquisition+Systems&rft.jtitle=arXiv.org&rft.au=Gao%2C+Yu&rft.au=Su%2C+Lutong&rft.au=Liang%2C+Hao&rft.au=Yue%2C+Yufeng&rft.date=2024-03-22&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422 |