A physics-informed deep learning liquid crystal camera with data-driven diffractive guidance

• Published in: Communications engineering Vol. 3; no. 1; pp. 46 - 10
• Main Authors: Shi, Jiashuo, Liu, Taige, Zhou, Liang, Yan, Pei, Wang, Zhe, Zhang, Xinyu
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 13.03.2024; Springer Nature B.V; Nature Portfolio
• Subjects: 639/166; 639/624/400; Algorithms; Back propagation; Cameras; Computation; Computer vision; Data processing; Datasets; Deep learning; Electric fields; Electrodes; Engineering; Environmental monitoring; Liquid crystals; Neural networks; Optical data processing; Parallel processing; Robotics; Target recognition; Tracking
• ISSN: 2731-3395; 2731-3395
• DOI: 10.1038/s44172-024-00191-7

Whether in the realms of computer vision, robotics, or environmental monitoring, the ability to monitor and follow specific targets amidst intricate surroundings is essential for numerous applications. However, achieving rapid and efficient target tracking remains a challenge. Here we propose an optical implementation for rapid tracking with negligible digital post-processing, leveraging an all-optical information processing. This work combines a diffractive-based optical nerual network with a layered liquid crystal electrical addressing architecture, synergizing the parallel processing capabilities inherent in light propagation with liquid crystal dynamic adaptation mechanism. Through a one-time effort training, the trained network enable accurate prediction of the desired arrangement of liquid crystal molecules as confirmed through numerical blind testing. Then we establish an experimental camera architecture that synergistically combines an electrically-tuned functioned liquid crystal layer with materialized optical neural network. With integrating the architecture into optical imaging path of a detector plane, this optical computing camera offers a data-driven diffractive guidance, enabling the identification of target within complex backgrounds, highlighting its high-level vision task implementation and problem-solving capabilities. Jiashuo Shi and colleagues build an integrated camera capable of tracking objects of interest. They use optical computing to arrange molecules in the liquid crystal mask for enhanced distinction between the object and background.