Physics-Informed Masked Autoencoder for active sparse imaging

• Published in: Scientific reports Vol. 14; no. 1; pp. 20078 - 8
• Main Authors: McEvoy, Luke, Tafone, Daniel, Sua, Yong Meng, Huang, Yuping
• Format: Journal Article
• Language: English
• Published: England Nature Publishing Group 29.08.2024; Nature Portfolio
• Subjects: Artificial intelligence; Data acquisition; Field programmable gate arrays; Gallium arsenide; Image processing; Lasers; Lidar; Physics; Quantum optics; Sensors; Signal processing; Single photon detection; Sparse reconstruction; Quantum optics; Sparse reconstruction; Artificial intelligence; Single photon detection
• ISSN: 2045-2322; 2045-2322
• DOI: 10.1038/s41598-024-71095-x

Imaging technology based on detecting individual photons has seen tremendous progress in recent years, with broad applications in autonomous driving, biomedical imaging, astronomical observation, and more. Comparing with conventional methods, however, it takes much longer time and relies on sparse and noisy photon-counting data to form an image. Here we introduce Physics-Informed Masked Autoencoder (PI-MAE) as a fast and efficient approach for data acquisition and image reconstruction through hardware implementation of the MAE (Masked Autoencoder). We examine its performance on a single-photon LiDAR system when trained on digitally masked MNIST data. Our results show that, with or less detected photons per pulse and down to 9 detected photons per pixel, it achieves high-quality image reconstruction on unseen object classes with 90% physical masking. Our results highlight PI-MAE as a viable hardware accelerator for significantly improving the performance of single-photon imaging systems in photon-starving applications.