Low-power, Continuous Remote Behavioral Localization with Event Cameras

• Published in: arXiv.org
• Main Authors: Hamann, Friedhelm, Ghosh, Suman, Ignacio Juarez Martinez, Hart, Tom, Kacelnik, Alex, Gallego, Guillermo
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 19.03.2024
• Subjects: Cameras; Computer vision; Data storage; Lighting; Power consumption; Power management; Remote monitoring; Remote observing; Weather

Researchers in natural science need reliable methods for quantifying animal behavior. Recently, numerous computer vision methods emerged to automate the process. However, observing wild species at remote locations remains a challenging task due to difficult lighting conditions and constraints on power supply and data storage. Event cameras offer unique advantages for battery-dependent remote monitoring due to their low power consumption and high dynamic range capabilities. We use this novel sensor to quantify a behavior in Chinstrap penguins called ecstatic display. We formulate the problem as a temporal action detection task, determining the start and end times of the behavior. For this purpose, we recorded a colony of breeding penguins in Antarctica for several weeks and labeled event data on 16 nests. The developed method consists of a generator of candidate time intervals (proposals) and a classifier of the actions within them. The experiments show that the event cameras' natural response to motion is effective for continuous behavior monitoring and detection, reaching a mean average precision (mAP) of 58% (which increases to 63% in good weather conditions). The results also demonstrate the robustness against various lighting conditions contained in the challenging dataset. The low-power capabilities of the event camera allow it to record significantly longer than with a conventional camera. This work pioneers the use of event cameras for remote wildlife observation, opening new interdisciplinary opportunities. https://tub-rip.github.io/eventpenguins/