Neural mixing for novel view synthesis

• Main Authors: GOLDMAN DANIEL, TSAI LYNN, MARTIN BRUALLA RICARDO, HEDMAN LARS PETER JOHANNES, HOPPE HUGUES HERVE
• Format: Patent
• Language: Chinese; English
• Published: 27.12.2022
• Subjects: CALCULATING; COMPUTING; COUNTING; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; PHYSICS

Systems and methods for receiving a plurality of input images, a plurality of depth images, and a plurality of view parameters to generate a virtual view of a target subject are described. The system and method may generate a plurality of warped images based on at least one of a plurality of input images, a plurality of view parameters, and a plurality of depth images. In response to providing a plurality of depth images, a plurality of view parameters, and a plurality of warped images to a neural network, the systems and methods may receive from the neural network a mix weight for assigning colors to pixels of a virtual view of a target subject, and may generate a composite image from the view parameters based on the mix weight and the virtual view. 描述了用于接收多个输入图像、多个深度图像和多个视图参数以生成目标被摄体的虚拟视图的系统和方法。该系统和方法可以基于多个输入图像、多个视图参数以及多个深度图像中的至少一个生成多个翘曲图像。响应于向神经网络提供多个深度图像、多个视图参数和多个翘曲图像，系统和方法可以从神经网络接收用于将颜色指配给目标被摄体的虚拟视图的像素的混合权重，并且可以基于混合权重和虚拟视图，根据视图参数生成合成图像。