Multi-Video Temporal Synchronization by Matching Pose Features of Shared Moving Subjects

Collaborative analysis of videos taken by multiple motion cameras from different and time-varying views can help solve many computer vision problems. However, such collaborative analysis usually requires the videos to be temporally synchronized, which can be inaccurate if we solely rely on camera cl...

Full description

Saved in:
Bibliographic Details
Published inIEEE International Conference on Computer Vision workshops pp. 2729 - 2738
Main Authors Wu, Xinyi, Wu, Zhenyao, Zhang, Yujun, Ju, Lili, Wang, Song
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.10.2019
Subjects
Cameras
Encoding
Feature extraction
Heating systems
Moving cameras
Synchronization
Three-dimensional displays
Video synchronization
Videos
Online AccessGet full text

Cover

More Information
Summary:Collaborative analysis of videos taken by multiple motion cameras from different and time-varying views can help solve many computer vision problems. However, such collaborative analysis usually requires the videos to be temporally synchronized, which can be inaccurate if we solely rely on camera clock. In this paper, we propose to address this problem based on video content. More specifically, if multiple videos cover the same moving persons, these subjects shall exhibit identical pose and pose change at each aligned time point across these videos. Based on this idea, we develop a new Synchronization Network (SynNet) which includes a feature aggregation module, a matching cost volume and several classification layers to infer the time offset between different videos by exploiting view-invariant human pose features. We conduct comprehensive experiments on SYN, SPVideo and MPVideo datasets. The results show that the proposed method can accurately synchronize multiple motion-camera videos collected in real world.
ISSN:2473-9944
DOI:10.1109/ICCVW.2019.00334