TensoRF: Tensorial Radiance Fields

• Published in: Computer Vision - ECCV 2022 Vol. 13692; pp. 333 - 350
• Main Authors: Chen, Anpei, Xu, Zexiang, Geiger, Andreas, Yu, Jingyi, Su, Hao
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer 2022; Springer Nature Switzerland
• Series: Lecture Notes in Computer Science
• ISBN: 3031198239; 9783031198236
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-031-19824-3_20

We present TensoRF, a novel approach to model and reconstruct radiance fields. Unlike NeRF that purely uses MLPs, we model the radiance field of a scene as a 4D tensor, which represents a 3D voxel grid with per-voxel multi-channel features. Our central idea is to factorize the 4D scene tensor into multiple compact low-rank tensor components. We demonstrate that applying traditional CANDECOMP/PARAFAC (CP) decomposition – that factorizes tensors into rank-one components with compact vectors – in our framework leads to improvements over vanilla NeRF. To further boost performance, we introduce a novel vector-matrix (VM) decomposition that relaxes the low-rank constraints for two modes of a tensor and factorizes tensors into compact vector and matrix factors. Beyond superior rendering quality, our models with CP and VM decompositions lead to a significantly lower memory footprint in comparison to previous and concurrent works that directly optimize per-voxel features. Experimentally, we demonstrate that TensoRF with CP decomposition achieves fast reconstruction (<30 $$<30$$ min) with better rendering quality and even a smaller model size (<4 $$<4$$ MB) compared to NeRF. Moreover, TensoRF with VM decomposition further boosts rendering quality and outperforms previous state-of-the-art methods, while reducing the reconstruction time (<10 $$<10$$ min) and retaining a compact model size (<75 $$<75$$ MB).