A Real-Time Multi-Stage Architecture for Pose Estimation of Zebrafish Head with Convolutional Neural Networks

• Published in: Journal of computer science and technology Vol. 36; no. 2; pp. 434 - 444
• Main Authors: Huang, Zhang-Jin, He, Xiang-Xiang, Wang, Fang-Jun, Shen, Qing
• Format: Journal Article
• Language: English
• Published: Singapore Springer Singapore 01.04.2021; Springer; Springer Nature B.V; School of Computer Science and Technology,University of Science and Technology of China,Hefei 230027,China; School of Data Science,University of Science and Technology of China,Hefei 230027,China; Anhui Province Key Laboratory of Software in Computing and Communication,Hefei 230027,China%School of Computer Science and Technology,University of Science and Technology of China,Hefei 230027,China%School of Computer Science and Technology,University of Science and Technology of China,Hefei 230027,China
• Subjects: Artificial Intelligence; Artificial neural networks; Body parts; Central processing units; Computer architecture; Computer Science; CPUs; Data Structures and Information Theory; Detectors; Information Systems Applications (incl.Internet); Neural networks; Neurophysiology; Object recognition; Pose estimation; Real time; Regular Paper; Software Engineering; Theory of Computation; Weight reduction; Zebrafish; zebrafish; real-time; convolutional neural network; pose estimation
• ISSN: 1000-9000; 1860-4749
• DOI: 10.1007/s11390-021-9599-5

In order to conduct optical neurophysiology experiments on a freely swimming zebrafish, it is essential to quantify the zebrafish head to determine exact lighting positions. To efficiently quantify a zebrafish head's behaviors with limited resources, we propose a real-time multi-stage architecture based on convolutional neural networks for pose estimation of the zebrafish head on CPUs. Each stage is implemented with a small neural network. Specifically, a light-weight object detector named Micro-YOLO is used to detect a coarse region of the zebrafish head in the first stage. In the second stage, a tiny bounding box refinement network is devised to produce a high-quality bounding box around the zebrafish head. Finally, a small pose estimation network named tiny-hourglass is designed to detect keypoints in the zebrafish head. The experimental results show that using Micro-YOLO combined with RegressNet to predict the zebrafish head region is not only more accurate but also much faster than Faster R-CNN which is the representative of two-stage detectors. Compared with DeepLabCut, a state-of-the-art method to estimate poses for user-defined body parts, our multi-stage architecture can achieve a higher accuracy, and runs 19x faster than it on CPUs.