Probing the 3D Awareness of Visual Foundation Models

• Published in: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 21795 - 21806
• Main Authors: Banani, Mohamed El, Raj, Amit, Maninis, Kevis-Kokitsi, Kar, Abhishek, Li, Yuanzhen, Rubinstein, Michael, Sun, Deqing, Guibas, Leonidas, Johnson, Justin, Jampani, Varun
• Format: Conference Proceeding
• Language: English
• Published: IEEE 16.06.2024
• Subjects: 3D Awareness; 3D Vision; Codes; Computer vision; Foundation Models; Image segmentation; Representation Learning; Solid modeling; Three-dimensional displays; Training; Visualization

Recent advances in large-scale pretraining have yielded visual foundation models with strong capabilities. Not only can recent models generalize to arbitrary images for their training task, their intermediate representations are useful for other visual tasks such as detection and segmentation. Given...