A Benchmark to Test Generalization Capabilities of Deep Learning Methods to Classify Severe Convective Storms in a Changing Climate

• Published in: Earth and space science (Hoboken, N.J.) Vol. 8; no. 9
• Main Authors: Molina, Maria J., Gagne, David John, Prein, Andreas F.
• Format: Journal Article
• Language: English
• Published: Hoboken John Wiley & Sons, Inc 01.09.2021; American Geophysical Union (AGU)
• Subjects: 21st century; benchmark; Boundary conditions; Climate; Climate change; convection; Deep learning; ENVIRONMENTAL SCIENCES; Kinematics; Machine learning; Morphology; Potential energy; Simulation; Thunderstorms; Weather; Weather forecasting; United States > US
• ISSN: 2333-5084; 2333-5084
• DOI: 10.1029/2020EA001490

This is a test case study assessing the ability of deep learning methods to generalize to a future climate (end of 21st century) when trained to classify thunderstorms in model output representative of the present‐day climate. A convolutional neural network (CNN) was trained to classify strongly rotating thunderstorms from a current climate created using the Weather Research and Forecasting model at high‐resolution, then evaluated against thunderstorms from a future climate and found to perform with skill and comparatively in both climates. Despite training with labels derived from a threshold value of a severe thunderstorm diagnostic (updraft helicity), which was not used as an input attribute, the CNN learned physical characteristics of organized convection and environments that are not captured by the diagnostic heuristic. Physical features were not prescribed but rather learned from the data, such as the importance of dry air at mid‐levels for intense thunderstorm development when low‐level moisture is present (i.e., convective available potential energy). Explanation techniques also revealed that thunderstorms classified as strongly rotating are associated with learned rotation signatures. Results show that the creation of synthetic data with ground truth is a viable alternative to human‐labeled data and that a CNN is able to generalize a target using learned features that would be difficult to encode due to spatial complexity. Most importantly, results from this study show that deep learning is capable of generalizing to future climate extremes and can exhibit out‐of‐sample robustness with hyperparameter tuning in certain applications. Plain Language Summary As temperatures and water vapor continue increasing due to climate change, models that were trained using past data may no longer perform with skill. Here, we explored whether the performance of a machine learning model was sensitive to a changing climate. The purpose of the machine learning model was to classify thunderstorms that were created using a high‐resolution numerical model into two groups: potentially severe thunderstorms and potentially non‐severe thunderstorms. Potentially severe thunderstorms were of interest because they have a greater likelihood of producing tornadoes and large hail, which cause billions of losses and dozens of fatalities every year. Results show that the machine learning model was able to classify thunderstorms with skill in both the present day and future climates partly due to the architecture of the machine learning model. We also explored the reasons behind the machine learning model's skill and found that it was able to learn thunderstorm characteristics and weather information from data. These results provide us with added confidence that machine learning models can learn physical relationships from weather and climate data and perform with skill in a changing climate in certain applications. Key Points A convolutional neural network can robustly classify convection in current and future climates Skillful classifications are based on learned thermodynamic and kinematic characteristics of thunderstorms Creating synthetic data with ground truth demonstrated to be a good alternative to the creation of human‐labeled data