Eyes on nature: Embedded vision cameras for terrestrial biodiversity monitoring

• Published in: Methods in ecology and evolution
• Main Authors: Darras, Kevin F. A., Balle, Marcel, Xu, Wenxiu, Yan, Yang, Zakka, Vincent G., Toledo‐Hernández, Manuel, Sheng, Dong, Lin, Wei, Zhang, Boyu, Lan, Zhenzhong, Fupeng, Li, Wanger, Thomas C.
• Format: Journal Article
• Language: English
• Published: 28.10.2024
• ISSN: 2041-210X; 2041-210X
• DOI: 10.1111/2041-210X.14436

Abstract We need comprehensive information to manage and protect biodiversity in the face of global environmental challenges, and artificial intelligence is required to generate that information from vast amounts of biodiversity data. Currently, vision‐based monitoring methods are heterogenous; they poorly cover spatial and temporal dimensions, overly depend on humans, and are not reactive enough for adaptive management. To mitigate these issues, we present a portable, modular, affordable and low‐power device with embedded vision for biodiversity monitoring of a wide range of terrestrial taxa. Our camera uses interchangeable lenses to resolve barely visible and remote targets, as well as customisable algorithms for blob detection, region‐of‐interest classification and object detection to automatically identify them. We showcase our system in six use cases from ethology, landscape ecology, agronomy, pollination ecology, conservation biology and phenology disciplines. Using the same devices with different setups, we discovered bats feeding on durian tree flowers, monitored flying bats and their insect prey, identified nocturnal insect pests in paddy fields, detected bees visiting rapeseed crop flowers, triggered real‐time alerts for waterfowl and tracked flower phenology over months. We measured classification accuracies (i.e. F 1‐scores) between 55% and 95% in our field surveys and used them to standardise observations over highly resolved time scales. Our cameras are amenable to situations where automated vision‐based monitoring is required off the grid, in natural and agricultural ecosystems, and in particular for quantifying species interactions. Embedded vision devices such as this will help addressing global biodiversity challenges and facilitate a technology‐aided agricultural systems transformation. 摘要 在全球环境面临挑战的背景下，我们需要获取全面的生物信息来管理和保护生物多样性，并且需要利用人工智能从大量的生物多样性数据中生成这些信息。目前，基于视觉的监测方法异质性较大，空间和时间维度覆盖不足，过于依赖人力，且无法及时响应变化的适应性管理策略。 为了解决这些问题，我们提出设计了一种便携、模块化、经济实惠且低功耗的设备。该设备具备嵌入式视觉功能，用于监测多种陆生物种的生物多样性。我们的相机使用可更换的镜头来捕捉几乎不可见的远距离目标，并通过定制化算法进行斑点检测、区域分类和目标检测来识别这些物种。我们展示了该系统在动物行为学、景观生态学、农学、传粉生态学、保护生物学和物候学等六个学科中的应用。 使用相同设备的不同自动化设置，我们监测到了蝙蝠在榴莲树花上觅食，监测了蝙蝠及其昆虫猎物的飞行活动，识别了稻田中的夜间害虫，检测了蜜蜂拜访油菜花，响应了对水禽的实时警报，并跟踪了花卉的物候变化。 我们在实地调查中测量了实现了55%到95%之间的分类准确率(即F1分数)，并利用这些数据在高分辨率时间尺度上标准化了观测结果。我们的相机适用于需要脱网脱网的自动化视觉监测的情况，尤其是在自然和农业生态系统中，我们的相机可以用于定量研究物种间的相互作用。像这样的嵌入式视觉监测设备将有助于应对全球生物多样性挑战，并促进技术辅助型的农业系统转型。