Persistent people tracking and face capture using a PTZ camera

• Published in: Machine vision and applications Vol. 27; no. 3; pp. 397 - 413
• Main Authors: Cai, Yinghao, Medioni, Gérard
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.04.2016; Springer Nature B.V
• Subjects: Cameras; Communications Engineering; Computer Science; Dynamical systems; Field of view; Forensic engineering; Image detection; Image Processing and Computer Vision; Manuals; Networks; Original Paper; Pattern Recognition; Pedestrians; Receiving; Schedules; Surveillance systems; Tracking; Tracking (position); Trajectories; Vision systems; Multiple target tracking; Pan–tilt–zoom camera; Pedestrian detection; Face detection; Face–person association
• ISSN: 0932-8092; 1432-1769
• DOI: 10.1007/s00138-016-0758-6

Pan–tilt–zoom (PTZ) camera is a powerful tool in far-field scenarios. However, most of the current PTZ surveillance systems require manual intervention to move the camera to the desired position. In this paper, we address the problem of persistent people tracking and face capture in uncontrolled scenarios using a single PTZ camera, which could prove most helpful in forensic applications. The system first detects and tracks pedestrians in zoomed-out mode. Then, according to a scheduler, the system selects a person to zoom in. In the zoomed-in mode, we detect a set of face images and solve the face–face association and face–person association problems. The system then zooms back out where tracking is continued as people re-appear in the view. The person–person association module associates the people on the schedule list with the people in the current view. The detected faces are associated with the corresponding people and trajectories. Due to the dynamic nature of our problem, e.g. the field of view of the camera changes because of the pan/tilt/zoom movement of the camera, all of the processes including receiving images from the camera and processing must be done in real time. To the best of our knowledge, the proposed method is the first to address the association of face images to people and trajectories using a single PTZ camera. Extensive experiments in challenging indoor and outdoor uncontrolled conditions demonstrate the effectiveness of the proposed system.