Fast hand and finger detection algorithm for interaction on smart display

• Published in: Displays Vol. 55; pp. 55 - 63
• Main Authors: Lee, Donghyeon, Kim, Jin-Sung, Lee, Hyuk-Jae
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.12.2018
• Subjects: Depth image; Finger detection; Hand detection; Human-computer interaction; Object detection; Smart display; Depth image; Human-computer interaction; Hand detection; Smart display; Object detection; Finger detection
• ISSN: 0141-9382; 1872-7387
• DOI: 10.1016/j.displa.2018.02.001

•A hand detection and tracking method for smart display interface is proposed.•Left and right edges of an object are used for hand detection in a depth image.•In two consecutive frames, the hand sample positions are compared for hand tracking.•There is no limitation in the number of hands for detection and tracking. This paper presents a fast and accurate hand detection and tracking algorithm for depth images using an object contour analysis for which pairs of horizontal edges of an object are used. The computational complexity for generating the pairs of edges and estimating the object shape is low. A shape analysis is performed in the raster scan order and an object which can be a finger is selected as the finger candidate during the line scan. The initial hand is detected by searching for an object with a shape that corresponds to the predefined shape of the initial hand in the entire depth range. Thus, there is no assumption of the hand location throughout the depth, and the cluttered background has little effect on the detection. This paper also proposes a low-complexity hand tracking method which achieves 97.11% in the detection rate and 10.39 ms in the average processing time for a frame, respectively, when a single hand is detected. For five hands, the average processing time is 21.15 ms. These results show that the proposed method can be applied to the real-time application of vision-based HCI especially for smart display.