METHOD AND SYSTEM FOR SIMULTANEOUS SCENE PARSING AND MODEL FUSION FOR ENDOSCOPIC AND LAPAROSCOPIC NAVIGATION

• Main Authors: KLUCKNER STEFAN, KAMEN ALI, CHEN TERRENCE
• Format: Patent
• Language: Chinese; English
• Published: 06.02.2018
• Subjects: CALCULATING; COMPUTING; COUNTING; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; PHYSICS

A method and system for scene parsing and model fusion in laparoscopic and endoscopic 2D/2.5D image data is disclosed. A current frame of an intra-operative image stream including a 2D image channel and a 2.5D depth channel is received. A 3D pre-operative model of a target organ segmented in pre-operative 3D medical image data is fused to the current frame of the intra-operative image stream. Semantic label information is propagated from the pre-operative 3D medical image data to each of a plurality of pixels in the current frame of the intra-operative image stream based on the fused pre-operative 3D model of the target organ, resulting in a rendered label map for the current frame of the intra-operative image stream. A semantic classifier is trained based on the rendered label map for thecurrent frame of the intra-operative image stream. 公开了种用于在腹腔镜和内窥镜2D/2.5D图像数据中进行场景解析和模型融合的方法和系统。接收包括2D图像通道和2.5D深度通道的术中图像流的当前帧。在术前3D医学图像数据中分割的目标器官的3D术前模型被融合到术中图像流的当前帧中。基于目标器官的融合术前3D模型，将来自术前3D医学图像数据的语义标签信息传播到术中图像流的当前帧中的多个像素中的每个