Comparative analysis of tracking and behavioral patterns between wild-type and genetically modified fruit flies using computer vision and statistical methods

• Published in: Behavioural processes Vol. 222; p. 105109
• Main Authors: Lu, Fei Ying, Liu, Xiang, Su, Hai Feng, Wang, Shuo Hong
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier B.V 01.10.2024
• Subjects: 3D reconstruction; Behavioral analysis; Filter tracking; Multi-object tracking; Multi-object tracking; 3D reconstruction; Filter tracking; Behavioral analysis
• ISSN: 0376-6357; 1872-8308; 1872-8308
• DOI: 10.1016/j.beproc.2024.105109

Collective animal behavior occurs in groups and swarms at almost every biological scale, from single-celled organisms to the largest animals on Earth. The intriguing mysteries behind these group behaviors have attracted many scholars, and while it is known that models can reproduce qualitative features of such complex behaviors, this requires data from real animals to demonstrate, and obtaining data on the exact features of these groups is tricky. In this paper, we propose the Hidden Markov Unscented Tracker (HMUT), which combines the state prediction capability of HMM and the high-precision nonlinear processing capability of UKF. This prediction-driven tracking mechanism enables HMUT to quickly adjust tracking strategies when facing sudden changes in target motion direction or rapid changes in speed, reducing the risk of tracking loss. Videos of fruit fly swarm movement in an enclosed environment are captured using stereo cameras. For the captured fruit fly images, the thresholded AKAZE algorithm is first used to detect the positions of individual fruit flies in the images, and the motion of the fruit flies is modeled using a multidimensional hidden Markov model (HMM). Tracking is then performed using the Unscented Kalman Filter algorithm to obtain the flight trajectories of the fruit flies in two camera views. Finally, 3D reconstruction of the trajectories in both views is achieved through polar coordinate constraints, resulting in 3D motion data of the fruit flies. Additionally, the efficiency and accuracy of the proposed algorithm are evaluated by simulating fruit fly swarm movement using the Boids algorithm. Finally, based on the tracked fruit fly flight data, behavioral characteristics of the fruit flies are analyzed from two perspectives. The first is a statistical analysis of the differences between the two behaviors. The second dimension involves clustering trajectory similarity using the DTW method based on fruit fly flight trajectories, further analyzing the similarity within clusters and differences between clusters. •In this paper, we propose the Hidden Markov Unscented Tracker (HMUT), which combines the state prediction capability of HMM and the high-precision nonlinear processing capability of UKF.•For the captured fruit fly images, the thresholded AKAZE algorithm is first used to detect the positions of individual fruit flies in the images, and the motion of the fruit flies is modeled using a multi-dimensional HMM.•Finally, 3D reconstruction of the trajectories in both view is achieved through polar coordinate constraints, resulting in 3D motion data of the fruit flies.•Finally, based on the tracked fruit flies flight data, behavioral characteristics of the fruit flies are analyzed from two perspectives.