New access modes of parallel memory subsystem for sub-pixel motion estimation

• Published in: Journal of real-time image processing Vol. 15; no. 2; pp. 279 - 296
• Main Authors: Jakovljević, Radomir, Berić, Aleksandar, van Dalen, Edwin, Milićev, Dragan
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.08.2018; Springer Nature B.V
• Subjects: Chips (memory devices); Computer architecture; Computer Graphics; Computer Science; Energy consumption; Image Processing and Computer Vision; Interpolation; Motion simulation; Multimedia Information Systems; Original Research Paper; Pattern Recognition; Pixels; Reading; Signal,Image and Speech Processing; SIMD (computers); State of the art; Energy consumption; Block access; Block-matching; Row access; Throughput; SIMD; Pixel interpolation
• ISSN: 1861-8200; 1861-8219
• DOI: 10.1007/s11554-014-0481-3

Accessing pixels in memory is a well-known bottleneck of SIMD (single instruction multiple data) processors in video/imaging. To tackle it, we propose new block and row access modes of parallel on-chip memory subsystem, which enable a higher processing throughput and lower energy consumption than the access modes of the state-of-the-art subsystems. The new access modes significantly reduce the number of on-chip memory accesses, and thereby accelerate one of key video/imaging kernels: sub-pixel block-matching motion estimation. The main idea is to exploit spatial overlaps of blocks/rows accessed for pixel interpolation, which are known at the subsystem design-time, and merge multiple accesses into a single one by accessing somewhat more pixels at a time than with other parallel memories. To avoid the need for a wider, and, therefore, more costly SIMD datapath, we propose new memory read operations that split all pixels accessed at a time into multiple SIMD-wide blocks/rows, in a convenient way for further processing. As a proof of concept, we describe a parametric, scalable, and cost-efficient architecture that supports the new access modes. The architecture is based on a previously proposed set of memory banks with multiple pixels per bank word, and a previously proposed shifted scheme for arranging pixels in the banks. We analytically and experimentally demonstrate advantages of this work on a case study of sub-pixel motion estimation for video frame-rate conversion. The implemented motion estimator processes 2160p video at 60 fps in real time, while clocked at 600 MHz. Compared to the implementations based on the state-of-the-art subsystems, this work enables 40–70 % higher throughput, consumes 17–44 % less energy and has similar silicon area and off-chip memory bandwidth costs. That is 1.8–2.9 times more efficient than the prior art, considering the throughput and all costs, i.e., consumption, area, and off-chip bandwidth. Such a higher efficiency is the result of the new access modes, which reduced the number of on-chip memory accesses by 1.6–2.1 times, and the cost-efficient architecture.