3D mouse pose from single-view video and a new dataset

• Published in: Scientific reports Vol. 13; no. 1; pp. 13554 - 12
• Main Authors: Hu, Bo, Seybold, Bryan, Yang, Shan, Sud, Avneesh, Liu, Yi, Barron, Karla, Cha, Paulyn, Cosino, Marcelo, Karlsson, Ellie, Kite, Janessa, Kolumam, Ganesh, Preciado, Joseph, Zavala-Solorio, José, Zhang, Chunlian, Zhang, Xiaomeng, Voorbach, Martin, Tovcimak, Ann E., Ruby, J. Graham, Ross, David A.
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 21.08.2023; Nature Publishing Group; Nature Portfolio
• Subjects: 631/114/2397; 631/1647/245; 631/1647/794; 639/705/1046; 639/705/117; 692/308; Accuracy; Algorithms; Animal models; Animals; Cameras; Datasets; Extremities; Foot; Genotype; Genotypes; Humanities and Social Sciences; Kinematics; Machine learning; Medical imaging; Mice; Models, Animal; multidisciplinary; Science; Science (multidisciplinary)
• ISSN: 2045-2322; 2045-2322
• DOI: 10.1038/s41598-023-40738-w

We present a method to infer the 3D pose of mice, including the limbs and feet, from monocular videos. Many human clinical conditions and their corresponding animal models result in abnormal motion, and accurately measuring 3D motion at scale offers insights into health. The 3D poses improve classification of health-related attributes over 2D representations. The inferred poses are accurate enough to estimate stride length even when the feet are mostly occluded. This method could be applied as part of a continuous monitoring system to non-invasively measure animal health, as demonstrated by its use in successfully classifying animals based on age and genotype. We introduce the Mouse Pose Analysis Dataset, the first large scale video dataset of lab mice in their home cage with ground truth keypoint and behavior labels. The dataset also contains high resolution mouse CT scans, which we use to build the shape models for 3D pose reconstruction.