Model Evaluation by Measurements from Collocated Remote Sensors in Complex Terrain

• Published in: Weather and forecasting Vol. 37; no. 10; pp. 1829 - 1853
• Main Authors: Pichugina, Yelena L., Banta, Robert M., Brewer, W. Alan, Kenyon, J., Olson, J. B., Turner, D. D., Wilczak, J., Baidar, S., Lundquist, J. K., Shaw, W. J., Wharton, S.
• Format: Journal Article
• Language: English
• Published: Boston American Meteorological Society 01.10.2022
• Subjects: Accuracy; Data analysis; Data processing; data processing/distribution; Energy; ENVIRONMENTAL SCIENCES; Error analysis; Errors; Evaluation; Forecast accuracy; Instruments; Laboratories; Measurement; Measurement methods; Measuring instruments; Model accuracy; model evaluation/performance; Model testing; Modelling; Parameterization; Physics; Quality control; Remote sensing; Remote sensors; renewable energy; Resolution; Sensors; Spatial variability; Spatial variations; Temporal resolution; Terrain models; Uncertainty; Variability; Weather forecasting; Wind; Wind farms; Wind measurement; Wind speed; Columbia River
• ISSN: 0882-8156; 1520-0434
• DOI: 10.1175/WAF-D-21-0214.1

Abstract Model improvement efforts involve an evaluation of changes in model skill in response to changes in model physics and parameterization. When using wind measurements from various remote sensors to determine model forecast accuracy, it is important to understand the effects of measurement-uncertainty differences among the sensors resulting from differences in the methods of measurement, the vertical and temporal resolution of the measurements, and the spatial variability of these differences. Here we quantify instrument measurement variability in 80-m wind speed during WFIP2 and its impact on the calculated errors and the change in error from one model version to another. The model versions tested involved updates in model physics from HRRRv1 to HRRRv4, and reductions in grid interval from 3 km to 750 m. Model errors were found to be 2–3 m s −1 . Differences in errors as determined by various instruments at each site amounted to about 10% of this value, or 0.2–0.3 m s −1 . Changes in model skill due to physics or grid-resolution updates also differed depending on the instrument used to determine the errors; most of the instrument-to-instrument differences were ∼0.1 m s −1 , but some reached 0.3 m s −1 . All instruments at a given site mostly showed consistency in the sign of the change in error. In two examples, though, the sign changed, illustrating a consequence of differences in measurements: errors determined using one instrument may show improvement in model skill, whereas errors determined using another instrument may indicate degradation. This possibility underscores the importance of having accurate measurements to determine the model error. Significance Statement To evaluate model forecast accuracy using remote sensing instruments, it is important to understand the effects of measurement uncertainties due to differences in the methods of measurement and data processing techniques, the vertical and temporal resolution of the measurements, and the spatial variability of these differences. In this study, three types of collocated remote sensing systems are used to quantify the impact of measurement variability on the magnitude of calculated errors and the change in error from one model version to another. The model versions tested involved updates in model physics from HRRRv1 to HRRRv4, and reductions in grid interval from 3 km to 750 m.