NEURAL RADIATION FIELD FOR VEHICLE

• Main Authors: VASUDEVAN, RAJEEV, ANDREAS KARLSSON, SRINIVASAN RAMANAGOPAL MADHAVAN, CHAKRAVARTHY, PUNAJ, JAIPURIA NIKITA, SKINNER KEVIN
• Format: Patent
• Language: Chinese; English
• Published: 06.02.2024
• Subjects: ANALOGOUS ARRANGEMENTS USING OTHER WAVES; CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION ORRERADIATION OF RADIO WAVES; MEASURING; PHYSICS; RADIO DIRECTION-FINDING; RADIO NAVIGATION; TESTING

The invention provides a neural radiation field for a vehicle. A computer includes a processor and a memory, and the memory stores instructions, the instructions are executable by the processor to jointly train geometric NeRF multilayer perception (MLP) and color NeRF MLP to model a scene using an occupancy grid map, camera data of the scene from a camera, and lidar data of the scene from a lidar; supervising the geometric NeRF MLP with the lidar data during the joint training period; and supervising the color NeRF MLP with the camera data during the joint training. The geometric NeRF MLP is a neural radiation field that models the geometry of the scene, and the color NeRF MLP is a neural radiation field that models the color of the scene. 本公开提供了"车辆的神经辐射场"。一种计算机包括处理器和存储器，并且所述存储器存储指令，所述指令可由所述处理器执行以使用占用网格地图、场景的来自相机的相机数据和所述场景的来自激光雷达的激光雷达数据来联合训练几何NeRF多层感知(MLP)和颜色NeRF MLP以对所述场景进行建模；在所述联合训练期间用所述激光雷达数据监督所述几何NeRF MLP；以及在所述联合训练期间用所述相机数据监督所述颜色NeRF MLP。几何NeRF MLP是对场景的几何形状进行建模的神经辐射场，并且颜色NeRF MLP是对场景的颜色进行建模的神经辐射场。