Zero-shot learning enables instant denoising and super-resolution in optical fluorescence microscopy

• Published in: Nature communications Vol. 15; no. 1; pp. 4180 - 15
• Main Authors: Qiao, Chang, Zeng, Yunmin, Meng, Quan, Chen, Xingye, Chen, Haoyu, Jiang, Tao, Wei, Rongfei, Guo, Jiabao, Fu, Wenfeng, Lu, Huaide, Li, Di, Wang, Yuwang, Qiao, Hui, Wu, Jiamin, Li, Dong, Dai, Qionghai
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 16.05.2024; Nature Publishing Group; Nature Portfolio
• Subjects: 14/19; 14/63; 14/69; 631/1647/245/2225; 631/1647/328/2238; 639/624/1107/328/2238; Algorithms; Animals; Artificial neural networks; Caenorhabditis elegans - embryology; Computer applications; Confocal microscopy; Data acquisition; Deep Learning; Diffraction; Embryos; Fluorescence; Fluorescence microscopy; Humanities and Social Sciences; Image enhancement; Image processing; Image Processing, Computer-Assisted - methods; Image resolution; Imaging, Three-Dimensional - methods; Light microscopy; Light sheets; Machine learning; Mice; Microscopes; Microscopy; Microscopy, Fluorescence - methods; multidisciplinary; Neural networks; Optical microscopy; Science; Science (multidisciplinary); Zero-shot learning
• ISSN: 2041-1723; 2041-1723
• DOI: 10.1038/s41467-024-48575-9

Computational super-resolution methods, including conventional analytical algorithms and deep learning models, have substantially improved optical microscopy. Among them, supervised deep neural networks have demonstrated outstanding performance, however, demanding abundant high-quality training data, which are laborious and even impractical to acquire due to the high dynamics of living cells. Here, we develop zero-shot deconvolution networks (ZS-DeconvNet) that instantly enhance the resolution of microscope images by more than 1.5-fold over the diffraction limit with 10-fold lower fluorescence than ordinary super-resolution imaging conditions, in an unsupervised manner without the need for either ground truths or additional data acquisition. We demonstrate the versatile applicability of ZS-DeconvNet on multiple imaging modalities, including total internal reflection fluorescence microscopy, three-dimensional wide-field microscopy, confocal microscopy, two-photon microscopy, lattice light-sheet microscopy, and multimodal structured illumination microscopy, which enables multi-color, long-term, super-resolution 2D/3D imaging of subcellular bioprocesses from mitotic single cells to multicellular embryos of mouse and C. elegans . The authors introduce ZS-DeconvNet, an unsupervised computational super-resolution method for multiple types of microscopes, that enhances image resolution by more than 1.5 times over the diffraction limit with 10 times lower fluorescence than regular superresolution imaging conditions.