Deep learning to extract the meteorological by‐catch of wildlife cameras

• Published in: Global change biology Vol. 30; no. 1; pp. e17078 - n/a
• Main Authors: Alison, Jamie, Payne, Stephanie, Alexander, Jake M., Bjorkman, Anne D., Clark, Vincent Ralph, Gwate, Onalenna, Huntsaar, Maria, Iseli, Evelin, Lenoir, Jonathan, Mann, Hjalte Mads Rosenstand, Steenhuisen, Sandy‐Lynn, Høye, Toke Thomas
• Format: Journal Article
• Language: English
• Published: England Blackwell Publishing Ltd 01.01.2024; Wiley
• Subjects: Accuracy; alpine ecology; Animal learning; automated monitoring; Autumn; bees; Biodiversity; Biological Sciences; Bombus; Bumblebees; bycatch; Cameras; Classification; Climate change; Climatology; Computer vision; data collection; Deep learning; Earth Sciences; Ecology; Ekologi; Environmental impact; Environmental Sciences; Global Changes; Hail; Hailstorms; Hemispheres; Microclimate; micrometeorology; Model accuracy; Mountains; Norway; proximal sensing; Sciences of the Universe; Seasonal variations; Snow; snow melt; solar radiation; South Africa; summer; Sunlight; temperature; time-lapse photography; Trifolium pratense; Wild plants; Wildlife; Norway; South Africa; snow melt; micrometeorology; automated monitoring; time-lapse photography; Trifolium pratense; alpine ecology; proximal sensing; bees; Micrometeorology; Alpine ecology; Snow melt; Time-lapse photography; Proximal sensing; Bees; Automated monitoring
• DOI: 10.1111/gcb.17078
• ISSN: 1354-1013; 1365-2486

Microclimate—proximal climatic variation at scales of metres and minutes—can exacerbate or mitigate the impacts of climate change on biodiversity. However, most microclimate studies are temperature centric, and do not consider meteorological factors such as sunshine, hail and snow. Meanwhile, remote cameras have become a primary tool to monitor wild plants and animals, even at micro‐scales, and deep learning tools rapidly convert images into ecological data. However, deep learning applications for wildlife imagery have focused exclusively on living subjects. Here, we identify an overlooked opportunity to extract latent, ecologically relevant meteorological information. We produce an annotated image dataset of micrometeorological conditions across 49 wildlife cameras in South Africa's Maloti‐Drakensberg and the Swiss Alps. We train ensemble deep learning models to classify conditions as overcast, sunshine, hail or snow. We achieve 91.7% accuracy on test cameras not seen during training. Furthermore, we show how effective accuracy is raised to 96% by disregarding 14.1% of classifications where ensemble member models did not reach a consensus. For two‐class weather classification (overcast vs. sunshine) in a novel location in Svalbard, Norway, we achieve 79.3% accuracy (93.9% consensus accuracy), outperforming a benchmark model from the computer vision literature (75.5% accuracy). Our model rapidly classifies sunshine, snow and hail in almost 2 million unlabelled images. Resulting micrometeorological data illustrated common seasonal patterns of summer hailstorms and autumn snowfalls across mountains in the northern and southern hemispheres. However, daily patterns of sunshine and shade diverged between sites, impacting daily temperature cycles. Crucially, we leverage micrometeorological data to demonstrate that (1) experimental warming using open‐top chambers shortens early snow events in autumn, and (2) image‐derived sunshine marginally outperforms sensor‐derived temperature when predicting bumblebee foraging. These methods generate novel micrometeorological variables in synchrony with biological recordings, enabling new insights from an increasingly global network of wildlife cameras. Cameras and artificial intelligence are revolutionizing data collection in ecology. However, wildlife cameras capture more than just wildlife; they also tell us about the environmental challenges faced by animals and plants. We trained a deep learning model to rapidly detect sunshine, snow and hail in 2 million images. The data help us tackle questions about pollination ecology and climate warming.