Ultrasound Despeckling With GANs and Cross Modality Transfer Learning

• Published in: IEEE access Vol. 12; pp. 45811 - 45823
• Main Authors: Vieira, Diogo Frois, Raposo, Afonso, Azeitona, Antonio, Afonso, Manya V., Pedro, Luis Mendes, Sanches, J.
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Deep learning; denoising; GANs; Generative adversarial networks; Image acquisition; Image reconstruction; Image resolution; Low noise; Magnetic resonance imaging; Medical imaging; Modal analysis; modality translation; Noise reduction; Speckle; Three-dimensional displays; Training; Ultrasonic imaging; Ultrasound
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2024.3381630

Ultrasound images are corrupted by a type of signal-dependent noise, called speckle, difficult to remove or attenuate with the classical denoising methods. On the contrary, structural Magnetic Resonance Imaging (MRI) is usually a high resolution low noise image modality that involves complex and expensive equipment and long acquisition times. Herein, a deep learning-based pipeline for speckle removal in B-mode ultrasound medical images, based on cross modality transfer learning, is proposed. The architecture of the system is based on a pix2pix Generative Adversarial Network (GAN), <inline-formula> <tex-math notation="LaTeX">D </tex-math></inline-formula>, able to denoise real B-mode ultrasound images by generating synthetic MRI-like versions by an image-to-image translation manner. The GAN <inline-formula> <tex-math notation="LaTeX">D </tex-math></inline-formula> was trained using two classes of image pairs: i) a set consisting of authentic MRI images paired with synthetic ultrasound images generated through a dedicated ultrasound simulator based on another GAN, <inline-formula> <tex-math notation="LaTeX">S </tex-math></inline-formula>, designed specifically for this purpose, and ii) a set comprising natural images paired with their corresponding noisy counterparts corrupted by Rayleigh noise. The denoising GAN proposed in this study demonstrates effective removal of speckle noise from B-mode ultrasound images. It successfully preserves the integrity of anatomical structures and avoids reconstruction artifacts, producing outputs that closely resemble typical MRI images. Comparative tests against other state-of-the-art methods reveal superior performance of the proposed denoising strategy across various reconstruction quality metrics.