Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes

• Published in: Nature methods Vol. 18; no. 6; pp. 678 - 687
• Main Authors: Chen, Jiji, Sasaki, Hideki, Lai, Hoyin, Su, Yijun, Liu, Jiamin, Wu, Yicong, Zhovmer, Alexander, Combs, Christian A., Rey-Suarez, Ivan, Chang, Hung-Yu, Huang, Chi Chou, Li, Xuesong, Guo, Min, Nizambad, Srineil, Upadhyaya, Arpita, Lee, Shih-Jong J., Lucas, Luciano A. G., Shroff, Hari
• Format: Journal Article
• Language: English
• Published: New York Nature Publishing Group US 01.06.2021; Nature Publishing Group
• Subjects: 631/114/1305; 631/1647/245/2225; Algorithms; Benchmarks; Bioinformatics; Biological Microscopy; Biological Techniques; Biomedical and Life Sciences; Biomedical Engineering/Biotechnology; Confocal microscopy; Deep Learning; Depletion; Fluorescence; Fluorescence microscopy; Image enhancement; Image Processing, Computer-Assisted; Image resolution; Image restoration; Life Sciences; Methods; Microscopes; Microscopy; Microscopy, Fluorescence - methods; Neural networks; Noise reduction; Photobleaching; Proteomics; Spatial discrimination; Spatial resolution; Stimulated emission; United States
• ISSN: 1548-7091; 1548-7105
• DOI: 10.1038/s41592-021-01155-x

We demonstrate residual channel attention networks (RCAN) for the restoration and enhancement of volumetric time-lapse (four-dimensional) fluorescence microscopy data. First we modify RCAN to handle image volumes, showing that our network enables denoising competitive with three other state-of-the-art neural networks. We use RCAN to restore noisy four-dimensional super-resolution data, enabling image capture of over tens of thousands of images (thousands of volumes) without apparent photobleaching. Second, using simulations we show that RCAN enables resolution enhancement equivalent to, or better than, other networks. Third, we exploit RCAN for denoising and resolution improvement in confocal microscopy, enabling ~2.5-fold lateral resolution enhancement using stimulated emission depletion microscopy ground truth. Fourth, we develop methods to improve spatial resolution in structured illumination microscopy using expansion microscopy data as ground truth, achieving improvements of ~1.9-fold laterally and ~3.6-fold axially. Finally, we characterize the limits of denoising and resolution enhancement, suggesting practical benchmarks for evaluation and further enhancement of network performance. Three-dimensional residual channel attention networks (RCAN) enable improved image denoising and resolution enhancement on volumetric time-lapse fluorescence microscopy data, allowing longitudinal super-resolution imaging of living samples.