Correlation Image Sensor for Algebraic Solution of Optical Flow

• Published in: IEEE journal of the Electron Devices Society Vol. 9; pp. 170 - 179
• Main Authors: Ando, Shigeru, Nagase, Masanori, Watanabe, Takashi, Kosugi, Tomohiko, Iida, Tetsuya
• Format: Journal Article
• Language: English
• Published: New York IEEE 2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algebra; Capacitors; CMOS; Correlation; Correlation detection; Electron devices; Fourier coefficient; Image sensor; Incident light; Interference; lock-in; Logic gates; Luminous intensity; Motion capture; Multiplication; Noise reduction; Object recognition; optical flow; Optical flow (image analysis); Optical imaging; Optical sensors; Reference signals; Velocity distribution; weighted integral method
• ISSN: 2168-6734; 2168-6734
• DOI: 10.1109/JEDS.2020.3048634

In this article, we describe several enhancements of a three-phase correlation image sensor (3PCIS) toward its uses for a direct algebraic method of optical flow detection, i.e., determining velocity field of image patterns for detecting, tracking, and three-dimensional recovery of objects in a scene. A recent theoretical study has provided an exact closed-form solution of the velocity from temporal Fourier integral measurements of a time-varying intensity distribution. An accurate analog multiplication and integration of incident light and reference signals required for this type of measurements is the inherent advantage of a 3PCIS. We fabricate and compare several types of correlation detection block. As continuous-value inputs of the algebraic solution, compensation operations of systematic sources of error are so designed to realize a near shot-noise-limited performance. For random telegraph noise and interference reduction, we show that a p-epi channel, deep p-well structure is the best 3P multiplier. Several experimental results of optical flow detection including an accuracy evaluation and the dense motion capture of a traffic scene are given.