Retinal spike train decoder using vector quantization for visual scene reconstruction

• Published in: Complex & intelligent systems Vol. 10; no. 3; pp. 3445 - 3458
• Main Authors: Ma, Kunwu, Raj, Alex Noel Joseph, Rajangam, Vijayarajan, Tjahjadi, Tardi, Liu, Minying, Zhuang, Zhemin
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.06.2024; Springer Nature B.V; Springer
• Subjects: Artificial intelligence; Artificial neural network; Artificial neural networks; Complexity; Computational Intelligence; Data Structures and Information Theory; Decoding; Direct conversion; Embedded codebook; Engineering; Feature extraction; Feature maps; Image processing; Image reconstruction; Neural networks; Original Article; Retinal images; Retinal impulse signal; Vector quantization; Artificial neural network; Vector quantization; Embedded codebook; Retinal impulse signal
• ISSN: 2199-4536; 2198-6053
• DOI: 10.1007/s40747-023-01333-8

The retinal impulse signal is the basic carrier of visual information. It records the distribution of light on the retina. However, its direct conversion to a scene image is difficult due to the nonlinear characteristics of its distribution. Therefore, the use of artificial neural network to reconstruct the scene from retinal spikes has become an important research area. This paper proposes the architecture of a neural network based on vector quantization, where the feature vectors of spike trains are extracted, compressed, and stored using a feature extraction and compression network. During the decoding process, the nearest neighbour search method is used to find the nearest feature vector corresponding to each feature vector in the feature map. Finally, a reconstruction network is used to decode a new feature map composed of matching feature vectors to obtain a visual scene. This paper also verifies the impact of vector quantization on the characteristics of pulse signals by comparing experiments and visualizing the characteristics before and after vector quantization. The network delivers promising performance when evaluated on different datasets, demonstrating that this research is of great significance for improving relevant applications in the fields of retinal image processing and artificial intelligence.