A Sparse and Spike‐Timing‐Based Adaptive Photoencoder for Augmenting Machine Vision for Spiking Neural Networks

• Published in: Advanced materials (Weinheim) Vol. 34; no. 48; pp. e2202535 - n/a
• Main Authors: Subbulakshmi Radhakrishnan, Shiva, Chakrabarti, Shakya, Sen, Dipanjan, Das, Mayukh, Schranghamer, Thomas F., Sebastian, Amritanand, Das, Saptarshi
• Format: Journal Article
• Language: English
• Published: Weinheim Wiley Subscription Services, Inc 01.12.2022
• Subjects: 2D materials; Biomimetics; Illumination; Integrated circuits; Logic circuits; Machine vision; Materials science; memtransistors; Neural networks; photoencoders; Photosensitivity; sparse spike‐based encoding; Spiking
• ISSN: 0935-9648; 1521-4095; 1521-4095
• DOI: 10.1002/adma.202202535

The representation of external stimuli in the form of action potentials or spikes constitutes the basis of energy efficient neural computation that emerging spiking neural networks (SNNs) aspire to imitate. With recent evidence suggesting that information in the brain is more often represented by explicit firing times of the neurons rather than mean firing rates, it is imperative to develop novel hardware that can accelerate sparse and spike‐timing‐based encoding. Here a medium‐scale integrated circuit composed of two cascaded three‐stage inverters and one XOR logic gate fabricated using a total of 21 memtransistors based on photosensitive 2D monolayer MoS2 for spike‐timing‐based encoding of visual information, is introduced. It is shown that different illumination intensities can be encoded into sparse spiking with time‐to‐first‐spike representing the illumination information, that is, higher intensities invoke earlier spikes and vice versa. In addition, non‐volatile and analog programmability in the photoencoder is exploited for adaptive photoencoding that allows expedited spiking under scotopic (low‐light) and deferred spiking under photopic (bright‐light) conditions, respectively. Finally, low energy expenditure of less than 1 µJ by the 2D‐memtransistor‐based photoencoder highlights the benefits of in‐sensor and bioinspired design that can be transformative for the acceleration of SNNs. A spike‐timing‐dependent sparse and adaptive photoencoder using an optoelectronic integrated circuit constructed using monolayer MoS2‐based 2D memtransistors is demonstrated. The photoencoder offers an energy and area efficient in‐sensor encoding solution that can be transformative for the acceleration of spiking neural networks.