ThunderPunch: A bare-hand, gesture-based, large interactive display interface with upper-body-part detection in a top view

• Published in: IEEE computer graphics and applications Vol. 38; no. 5; pp. 100 - 111
• Main Authors: Kim, Daehwan, Kim, Hye-mi, Kim, Hang-Kee, Park, Su-Ran, Lee, Ki-Suk, Kim, Ki-Hong
• Format: Magazine Article
• Language: English
• Published: United States IEEE 01.09.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Cameras; computer graphics; Estimation; Gesture recognition; Hardware; Hybrid systems; Image edge detection; large interactive display interface; Three-dimensional displays; upper body part detection
• ISSN: 0272-1716; 1558-1756
• DOI: 10.1109/MCG.2018.053491734

We present a new bare-hand gesture interface for large-screen interaction in which multiple users can participate simultaneously and interact with virtual content directly. To better reflect the intent of our new interface, we have created a new type of hardware system with a large hybrid display, named ThunderPunch. Unlike the conventional method, which involves positioning the camera in front, the cameras are mounted on the ceiling so that they avoid covering the large screen. To achieve bare-hand interaction in this hardware structure, we propose real-time algorithms that detect multiple body poses and recognize punching and touching gestures from top-view depth images. A pointing and touching test shows that the proposed algorithm is usable and that it outperforms other algorithms. In addition, we created a game to make the best use of the proposed system.