Automatic Scoring of Synchronization from Fingers Motion Capture and Music Beats

Thanks to various technological progresses such as musical rhythm estimation and motion capture systems, the evaluation of synchronization performances between motion and music beats is today possible. In this paper, we propose an innovative playful method to assess synchronization between hand moti...

Full description

Saved in:
Bibliographic Details
Published inImage Analysis and Processing. ICIAP 2022 Workshops pp. 235 - 245
Main Authors Bayd, Hamza, Guyot, Patrice, Bardy, Benoit, Slangen, Pierre R. L.
Format Book Chapter
LanguageEnglish
Published Cham Springer International Publishing
SeriesLecture Notes in Computer Science
Subjects
Dynamic time warping
Finger
Hand
Motion capture
Music beats
Synchronization
Online AccessGet full text

Cover

More Information
Summary:Thanks to various technological progresses such as musical rhythm estimation and motion capture systems, the evaluation of synchronization performances between motion and music beats is today possible. In this paper, we propose an innovative playful method to assess synchronization between hand motion and music. In this application, the hand gestures are tracked from live webcam motion capture based on MediaPipe open-source framework. Musical beats are estimated from the Librosa library. The synchronization between finger motion and beats is then computed from the dynamic time warping algorithm. A preliminary study was conducted with two different songs mixed with tempo beats at different intensities. The results are encouraging and show different levels of performance according to the tempo of the songs.
ISBN:9783031133206
303113320X
ISSN:0302-9743
1611-3349
DOI:10.1007/978-3-031-13321-3_21