Programmable Tanh- and ELU-Based Photonic Neurons in Optics-Informed Neural Networks

• Published in: Journal of lightwave technology Vol. 42; no. 10; pp. 3652 - 3660
• Main Authors: Kovaios, Stefanos, Pappas, Christos, Moralis-Pegios, Miltiadis, Tsakyridis, Apostolos, Giamougiannis, George, Kirtas, Manos, Van Kerrebrouck, Joris, Coudyzer, Gertjan, Yin, Xin, Passalis, Nikolaos, Tefas, Anastasios, Pleros, Nikos
• Format: Journal Article
• Language: English
• Published: New York IEEE 15.05.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Adaptive optics; Artificial neural networks; Integrated circuits; Machine learning; Neural networks; Neurons; Noise aware training; nonlinear activation function; Nonlinear optics; Optical communication; optical neural networks; opto-electronic activation function; Optoelectronic devices; Performance evaluation; Photonics; Training; Wavelength division multiplexing
• ISSN: 0733-8724; 1558-2213
• DOI: 10.1109/JLT.2024.3366711

We demonstrate an integrated opto-electronic (ΟΕ) device that can be programmed to provide a set of nonlinear activation functions (AFs) and present its operation within programmable tanh- and ELU-based photonic neurons at line rates up to 10 GBd. The OE activation module provides a set of well-known activation functions that are typically used in DL training models, including the tanh-, ELU- and inverted ELU-like functions. Its performance is experimentally evaluated when incorporated in a 4-input wavelength division multiplexed (WDM) photonic neuron and operating with non-deterministic data patterns, providing "noisy" tanh, ELU and inverted ELU AFs with an error-distribution that has in all cases a standard deviation of <0.49. We also evaluate the trainability of these "noisy" AFs and present for the first time an optics-informed training framework that incorporates the pattern-induced AF variations into the training process, yielding the first noise-aware training scheme where the noise emerges at the nonlinear AF NN segment. The performance analysis of the optics-informed training framework for all three AFs was carried out via Deep Learning setups suitable for classifying the Fashion MNIST and the CIFAR-10 datasets. This analysis has shown that the employment of traditional training schemes leads to significant accuracy degradations, which can be, however, almost completely waived when employing the optics-informed training framework, leading to accuracy values that are almost identical to the reference accuracy values obtained when ideal and noise-less AFs are used.