All-analog photoelectronic chip for high-speed vision tasks

• Published in: Nature (London) Vol. 623; no. 7985; pp. 48 - 57
• Main Authors: Chen, Yitong, Nazhamaiti, Maimaiti, Xu, Han, Meng, Yao, Zhou, Tiankuang, Li, Guangpu, Fan, Jingtao, Wei, Qi, Wu, Jiamin, Qiao, Fei, Fang, Lu, Dai, Qionghai
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group 02.11.2023; Nature Publishing Group UK
• Subjects: Accuracy; Analog to digital converters; Classification; Computation; Data compression; Driving ability; Energy consumption; Energy efficiency; Feature extraction; Latency; Light; Light effects; Neural networks; Noise; Optical encoders; Optics; Optoelectronics; Photoelectric effect; Power consumption; Vision; Wearable technology
• ISSN: 0028-0836; 1476-4687
• DOI: 10.1038/s41586-023-06558-8

Abstract Photonic computing enables faster and more energy-efficient processing of vision data 1–5 . However, experimental superiority of deployable systems remains a challenge because of complicated optical nonlinearities, considerable power consumption of analog-to-digital converters (ADCs) for downstream digital processing and vulnerability to noises and system errors 1,6–8 . Here we propose an all-analog chip combining electronic and light computing (ACCEL). It has a systemic energy efficiency of 74.8 peta-operations per second per watt and a computing speed of 4.6 peta-operations per second (more than 99% implemented by optics), corresponding to more than three and one order of magnitude higher than state-of-the-art computing processors, respectively. After applying diffractive optical computing as an optical encoder for feature extraction, the light-induced photocurrents are directly used for further calculation in an integrated analog computing chip without the requirement of analog-to-digital converters, leading to a low computing latency of 72 ns for each frame. With joint optimizations of optoelectronic computing and adaptive training, ACCEL achieves competitive classification accuracies of 85.5%, 82.0% and 92.6%, respectively, for Fashion-MNIST, 3-class ImageNet classification and time-lapse video recognition task experimentally, while showing superior system robustness in low-light conditions (0.14 fJ μm −2 each frame). ACCEL can be used across a broad range of applications such as wearable devices, autonomous driving and industrial inspections.