High resolution neural rendering

• Main Authors: GARBIN STEPHEN J, MARK ANDREW JOHNSON, KOWALSKI MARK A
• Format: Patent
• Language: Chinese; English
• Published: 07.11.2023
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; PHYSICS

Methods and systems are provided for training a machine learning model to generate density values and radiation components based on location data and a weighting scheme associated with a particular view direction based on directional data to calculate final RGB values for each point along a plurality of camera rays. The position data and the direction data are extracted from a set of training images of a particular static scene. Radiation components, density values, and weighting schemes are cached to achieve efficient image data processing to perform volume rendering for each point of sampling. A novel viewpoint of the static scene is generated based on the volume rendering of each point sampled. 提供了用于训练机器学习模型，以基于位置数据以及基于方向数据的与特定视图方向相关联的加权方案来生成密度值和辐射分量，从而计算沿着多个相机光线的每个点的最终RGB值的方法和系统。位置数据和方向数据是从特定静态场景的一组训练图像中提取的。辐射分量、密度值和加权方案被缓存，以实现高效的图像数据处理，从而对采样的每个点执行体积渲染。基于采样的每个点的体积渲染生成静态场景的新颖视点。