Cross-modal image generation and detection method and device

• Main Authors: MA BOYUAN, ZHAO JIHUAI, BAN XIAOJUAN, WANG XIAOKUN
• Format: Patent
• Language: Chinese; English
• Published: 01.11.2022
• Subjects: CALCULATING; COMPUTING; COUNTING; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; PHYSICS

The invention relates to a cross-modal image generation and detection method and device, belongs to the technical field of image processing and artificial intelligence, and can realize cross-modal generation of medical images and complete target detection. The method comprises the following steps: S1, acquiring an image set of more than two modes and performing registration; s2, fusing the image set to obtain a first multi-modal fusion image; s3, training a cross-modal image generation model by using the first multi-modal fusion image and the single-modal image corresponding to the first multi-modal fusion image; s4, inputting a single-modal image corresponding to the first multi-modal fusion image into a cross-modal image generation model to obtain a second multi-modal fusion image; s5, inputting the second multi-modal fusion image and the corresponding first multi-modal fusion image into a target detection model for training; and S6, performing end-to-end tuning on the model to obtain a final cross-modal im