VIRTUAL VIEWPOINT IMAGE GENERATION SYSTEM, IMAGE PROCESSING DEVICE, IMAGE GENERATION DEVICE, CONTROL METHODS THEREFOR, AND PROGRAM

• Main Author: FURUKAWA TAKASHI
• Format: Patent
• Language: English; Japanese
• Published: 13.05.2022
• Subjects: CALCULATING; COMPUTING; COUNTING; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; PHYSICS

To provide a technique for generating an appropriate image quality deterioration of a virtual viewpoint image according to a change in lighting condition.SOLUTION: A virtual viewpoint image generation system generates a virtual viewpoint image on the basis of three-dimensional shape information of a foreground, which is a subject based on a plurality of captured images acquired by a plurality of imaging devices, color information of the foreground, three-dimensional shape information of a background in an imaging space of the plurality of imaging devices, unlike the foreground, color information of the background, and a designated virtual viewpoint. The virtual viewpoint image generation system determines whether or not a lighting condition has changed beyond a predetermined criterion on the basis of at least one of the plurality of captured images, and adjusts the color information of at least one of the foreground and the background so that lighting conditions of the color information of the foreground and the background used for generating the virtual viewpoint image are brought closer, when it is determined that the lighting condition has changed.SELECTED DRAWING: Figure 1A 【課題】照明条件の変化に応じた適切な仮想視点画像の画質劣化を生成する技術を提供する。【解決手段】仮想視点画像生成システムは、複数の撮像装置により取得された複数の撮像画像に基づく被写体である前景の３次元形状情報と、前景の色情報と、前景とは異なり複数の撮像装置の撮像空間における背景の３次元形状情報と、背景の色情報と、指定された仮想視点とに基づいて、仮想視点画像を生成する。仮想視点画像生成システムは、複数の撮像画像の少なくとも１つに基づいて、照明条件に所定の基準を超える変化が生じたか否かを判定し、照明条件に変化が生じたと判定された場合に、仮想視点画像の生成に用いられる前景および背景の色情報の照明条件を近づけるように、前景および背景の色情報の少なくとも一方を調整する。【選択図】 図１Ａ