RGB-D ergonomic assessment system of adopted working postures

• Published in: Applied ergonomics Vol. 80; pp. 75 - 88
• Main Authors: Abobakr, Ahmed, Nahavandi, Darius, Hossny, Mohammed, Iskander, Julie, Attia, Mohammed, Nahavandi, Saeid, Smets, Marty
• Format: Journal Article
• Language: English
• Published: England Elsevier Ltd 01.10.2019
• Subjects: Adult; Biomechanical Phenomena; CNN; ConvNet; Deep learning; Ergonomics; Ergonomics - methods; Female; Humans; Male; Manufacturing and Industrial Facilities; MSDs; Musculoskeletal Diseases - diagnosis; Musculoskeletal Diseases - etiology; Occupational Diseases - diagnosis; Occupational Diseases - etiology; Posture - physiology; Posture analysis; RGB-D; Risk Assessment - methods; Risk Factors; RULA; Work - physiology; RGB-D; Deep learning; MSDs; CNN; Ergonomics; ConvNet; Posture analysis; RULA
• ISSN: 0003-6870; 1872-9126
• DOI: 10.1016/j.apergo.2019.05.004

Ensuring a healthier working environment is of utmost importance for companies and global health organizations. In manufacturing plants, the ergonomic assessment of adopted working postures is indispensable to avoid risk factors of work-related musculoskeletal disorders. This process receives high research interest and requires extracting plausible postural information as a preliminary step. This paper presents a semi-automated end-to-end ergonomic assessment system of adopted working postures. The proposed system analyzes the human posture holistically, does not rely on any attached markers, uses low cost depth technologies and leverages the state-of-the-art deep learning techniques. In particular, we train a deep convolutional neural network to analyze the articulated posture and predict body joint angles from a single depth image. The proposed method relies on learning from synthetic training images to allow simulating several physical tasks, different body shapes and rendering parameters and obtaining a highly generalizable model. The corresponding ground truth joint angles have been generated using a novel inverse kinematics modeling stage. We validated the proposed system in real environments and achieved a joint angle mean absolute error (MAE) of 3.19±1.57∘ and a rapid upper limb assessment (RULA) grand score prediction accuracy of 89% with Kappa index of 0.71 which means substantial agreement with reference scores. This work facilities evaluating several ergonomic assessment metrics as it provides direct access to necessary postural information overcoming the need for computationally expensive post-processing operations. •This paper proposes a vision-based semi-automated RULA ergonomic posture assessment system using deep learning techniques.•The proposed method analyzes the posture holistically and estimates body joint angles directly from a single depth image.•We presented a novel inverse kinematic modeling process to obtain ground truth joint angles from motion capture data.•This allows training learning algorithms to estimate joint angles from input images acquired using different modalities.•The developed system supports Kinect and ASUS Xtion depth cameras and does not rely on skeleton data from the Kinect SDK.•The holistic posture analysis approach ensures robustness to different forms of occlusions.