Road Weather Forecast Performance, Winter 2017-2018

To measure our forecast quality, we take a strict user perspective. We apply scores which are scientifically sound while still being intuitively understandable. We inspect weather situations where road services are most vulnerable. We accept the limitations of road weather observations for verification purposes. Following the suggestion of the Institution of Civil Engineers, we compare the forecasted road frost against the actual observed. This is applied during critical nights with minimum road surface temperature below 3 °C, in a contingency table as given in Table 1.

 

Frost forecast

No frost forecast

Frost occurred

F/F (Protection cost)

NF/F (Damage cost)

No frost occurred

F/NF (Protection cost)

NF/NF (Correct rejection)

Table 1 Two-by-two contingency table as suggested by Institution of Civil Engineers

The contingency table provides input for quality scores like

  • Percentage Correct = [(F/F + NF/NF) / (total number of critical nights) * 100]
  • Miss Rate = (NF/F) / (F/F + NF/F) * 100
  • False Alarm Ratio = (F/NF) / (F/F + F/NF) * 100

Figure 1 gives a high-level summary of MeteoGroup road frost forecast accuracy during the challenging road winter season 2017/2018. Averaged over all countries, more than 94% of our road frost forecasts have been correct. Figure 3 shows the distribution of these results over the total number of investigated stations.

In detail, evening forecasts as issued between 1 October 2017 and 1 April 2018, have been compared with observations made by more than 1.500 road weather stations. One third of these stations is located in the UK, another 400 in Germany, and about 200 each in the Netherlands, Switzerland, Liechtenstein, and Spain. Additionally, a smaller set of 60 French stations was used for our quality assessment.

Regional performance is of course impacted by local specifics, as well as by the dominant weather patterns of the past season. In Spain, most road weather stations are located on the Iberian plateaus at elevations between 800 m and 1500 m above sea level. Up to 145 nights have experienced critical road frost conditions. Our forecasters supported road services during several major snowfall events, which brought up to 70 cm of fresh snow and hauled long into late April. In Switzerland, most road stations are located in narrow Alpine valleys which are between 400 and 700 m above sea level.  Local micro-climates have a strong influence on the forecast performance and resulted in up to 160 critical nights during the past season. In the Netherlands, MeteoGroup provided excellent forecast accuracy, this past winter. This is only possible with expert knowledge of local wind systems alongside the shores and the specifics of bridges, as well as a close monitoring of clearing clouds. Figure 2 shows detailed results for the Netherlands, where on average 97% of the 80-100 critical nights were forecasted correctly. In Germany, potential forecast accuracy is determined by mountainous terrain. Half of the German road weather stations are located at elevations over 200 m above sea level, a quarter over 400 m above sea level. Additionally, a high number (nearly 40%) of road weather stations are installed on bridges, which react more sensitively to unforeseen changes in the atmospheric conditions. In the UK, both mountains and sea, and some of the most challenging weather patterns of the past decade, had an influence on the forecast accuracy during this winter season. Depending on the location, 65 to 140 nights have had critical road frost conditions. Only an excellent forecasting model and expert forecasters could achieve an average accuracy of 95% correct forecasts; with less than 5% missed road frost nights and only 6% of false alarms. On March 1st, 2018, the storm Emma brought widespread heavy snow and drift into the south-west England, followed by and alternating with freezing rain. it gave our weather rooms the chance to prove high forecasting skills and accuracy. It has already entered the annals of British road services.


Figure 1. MeteoGroup Road weather forecast accuracy during season 2017/18. On average, 94% of MeteoGroup’s road frost forecasts for the next night have been correct, with a false alarm ratio of less than 5%.


Figure 2. Percentage correct road frost forecasts by MeteoGroup during the 2017/18 season in the Netherlands. Between 80 and 100 nights during the season have been critical for road frost (including bridges), and on average 97% of them have been forecasted correctly. The average false alarm ratio was less than 2%.


Figure 3. Distribution of correct forecast percentage amongst the inspected ~1500 stations.

Correct forecasts distributionFor resource scheduling by winter road services, the major decision support is provided by the noon forecast issue. MeteoGroup updates forecasts every hour, incorporating fresh precipitation radar soundings as well as new reports from thousands of weather stations and many more observations to capture the latest atmospheric development. Until the evening, an average gain of 1% in temperature forecast accuracy is achieved. For snowfall and freezing rain, the accuracy gain is even higher, particularly during capricious atmospheric conditions.

end of season report

Please click here for the full End-of Season Report with detailed information for individual countries and an evaluation of the MeteoGroup forecast quality.

 

Learn more about MeteoGroup's full Winter Maintenance Services

 

During the winter season 2017/18, on average, 94% of MeteoGroup’s road frost forecasts for the next night have been correct, with a false alarm ratio of less than 5%