Extracting Latent Variables From Forecast Ensembles and Advancements in Similarity Metric Utilizing Optimal Transport

• Published in: Journal of geophysical research. Machine learning and computation Vol. 1; no. 2
• Main Author: Nishizawa, S.
• Format: Journal Article
• Language: English
• Published: 01.06.2024
• Subjects: dimension reduction; ensemble forecast; location errors; optimal transport; similarity metric
• ISSN: 2993-5210; 2993-5210
• DOI: 10.1029/2023JH000112

This study presents a novel methodology for extracting latent variables from high‐dimensional sparse data, particularly emphasizing spatial distributions such as precipitation distribution. This approach utilizes multidimensional scaling with a distance matrix derived from a new similarity metric, the Unbalanced Optimal Transport Score (UOTS). UOTS effectively captures discrepancies in spatial distributions while preserving physical units. This is similar to mean absolute error, however it considers location errors, providing a more robust measure crucial for understanding differences between observations, forecasts, and ensembles. Probability distribution estimation of these latent variables enhances the analytical utility, quantifying ensemble characteristics. The adaptability of the method to spatiotemporal data and its ability to handle errors suggest its potential as a promising tool for diverse research applications. Plain Language Summary This study introduces a new method to understand weather patterns by simplifying complex data. A mathematical technique was developed to efficiently identify hidden information from patterns. This assists meteorologists in understanding the weather with greater accuracy. This method simplifies weather data by highlighting the essential similarities and differences between weather patterns, making it easier for scientists to interpret and use the resultant data effectively. This study offers a new and efficient way to make sense of vast weather data, benefiting meteorological research, and potentially improving weather forecasting. The technique contributes to the meteorological field, in addition it also contributes to various fields with sparse distribution data. Key Points Novel method reveals hidden information from spatial ensemble data for understanding probability distributions Technique extracts essential similarities and differences in sparse distributions, aiding interpretation for improved analysis Approach is adaptable to different data types, making it promising for diverse scientific fields