Enhanced skeleton and face 3D data for person re-identification from depth cameras

• Published in: Computers & graphics Vol. 79; pp. 69 - 80
• Main Authors: Pala, Pietro, Seidenari, Lorenzo, Berretti, Stefano, Del Bimbo, Alberto
• Format: Journal Article
• Language: English
• Published: Oxford Elsevier Ltd 01.04.2019; Elsevier Science Ltd
• Subjects: 3D face; Body skeleton; Boosting; Cameras; Data acquisition; Depth camera; Face recognition; Feature extraction; Feature recognition; Geometric accuracy; Identification; Person re-identification; Photometry; Redundancy; Boosting; Depth camera; Person re-identification; 3D face; Body skeleton
• ISSN: 0097-8493; 1873-7684
• DOI: 10.1016/j.cag.2019.01.003

•Person re-identification approach from depth cameras.•Cumulating 3D skeletal and 3D face models across temporal sequences.•Weak classification of 3D skeletal features.•3D face reconstruction and refinement from depth frames.•Weighted combination of 3D skeletal and 3D face features. [Display omitted] Person re-identification is typically performed using 2D still images or videos, where photometric appearance is the main visual cue used to discover the presence of a target subject when switching from different camera views across time. This invalidates any application where a person may change dress across subsequent acquisitions as can be the case of patients monitoring at home. Differently from RGB data, 3D information as acquired by depth cameras can open the way to person re-identification based on biometric cues such as distinguishing traits of the body or face. However, the accuracy of skeleton and face geometry extracted from depth data is not always adequate to enable person recognition, since both these features are affected by the pose of the subject and the distance from the camera. In this paper, we propose a method to derive a robust skeleton representation from a depth sequence and to complement it with a highly discriminative face feature. This is obtained by selecting skeleton and face samples based on their quality and using the temporal redundancy across the sequence to derive and refine cumulated models for both of them. Extracting skeleton and face features from such cumulated models and combining them for the recognition allow us to improve rank-1 re-identification accuracy compared to individual cues. A comparative evaluation on three benchmark datasets also shows results at the state-of-the-art.