Computational 3D topographic microscopy from terabytes of data per sample

• Main Authors: Zhou, Kevin C, Harfouche, Mark, Zheng, Maxwell, Jönsson, Joakim, Lee, Kyung Chul, Appel, Ron, Reamey, Paul, Doman, Thomas, Saliu, Veton, Horstmeyer, Gregor, Horstmeyer, Roarke
• Format: Journal Article
• Language: English
• Published: 05.06.2023
• Subjects: Computer Science - Computer Vision and Pattern Recognition; Computer Science - Learning; Physics - Optics
• DOI: 10.48550/arxiv.2306.02634

We present a large-scale computational 3D topographic microscope that enables 6-gigapixel profilometric 3D imaging at micron-scale resolution across $>$110 cm$^2$ areas over multi-millimeter axial ranges. Our computational microscope, termed STARCAM (Scanning Topographic All-in-focus Reconstruction with a Computational Array Microscope), features a parallelized, 54-camera architecture with 3-axis translation to capture, for each sample of interest, a multi-dimensional, 2.1-terabyte (TB) dataset, consisting of a total of 224,640 9.4-megapixel images. We developed a self-supervised neural network-based algorithm for 3D reconstruction and stitching that jointly estimates an all-in-focus photometric composite and 3D height map across the entire field of view, using multi-view stereo information and image sharpness as a focal metric. The memory-efficient, compressed differentiable representation offered by the neural network effectively enables joint participation of the entire multi-TB dataset during the reconstruction process. To demonstrate the broad utility of our new computational microscope, we applied STARCAM to a variety of decimeter-scale objects, with applications ranging from cultural heritage to industrial inspection.