How it's made: The Non-Meteorologist’s Guide to Weather Data Quality Control

How It’s Made series is a 20-part series, giving you a look under the hood of weather forecasting, sharing the knowledge and experience of the weather experts.

Today we’re exploring Category Four: Quality Control & Data Management. In particular, we’re looking at why quality control is so important when creating your weather forecast.

Missed any of the earlier posts in the series? Check out the How It’s Made series here.

Make sure you don’t miss out on the rest of the series. Sign up for the weekly How It’s Made Update. Stay informed and get weekly updates, delivered straight to your inbox.
 

Sign Up Now

 

If the weather is important to your business, you’re going to care about the quality of your data. Every element of the weather forecasting process is rigorously and objectively monitored by the experts to improve data accuracy and your confidence in the data. They use verification methods recommended by the World Meteorology Organization to ensure standards are maintained.

It requires a lot of investment; meteorological researchers and their expertise are vital to quality control. As well, quality control requires supporting technology to ensure all the necessary checks can happen.

 

"When processing over 20.000 weather stations on the globe with hourly updates of the observation and the forecast, and adjusting all these forecasts with fresh data from several weather models up to 360h, operating a timely and well-designed automated quality control is a must"

Wim van den Berg,
Senior meteorological consultant | Weather Systems Team
MeteoGroup

 

What is quality control?

Quality control is a key step for all elements of the weather forecasting process. It’s a way of maintaining standards, by testing the outputs against the expected output. It typically takes place in two stages:

  • Incoming observation data, which is checked for accuracy, completeness, and irregularities; and
  • Forecast performance, which is verified before being used for reporting, and also provides input for both learning and improving forecast systems.

 

How the experts apply quality controls to observations

Observations are subject to intensive quality control procedures. For instance, weather stations are checked to remove errors to standardize the data for universal use and provide quality-checked accurate historical observation data. Algorithms are used to declutter the radar data and reduce the number of false precipitation signals.

 

How the experts apply quality controls to weather models

The experts draw data from multiple weather models to improve the accuracy of their forecast systems. This includes monitoring performance and establishing a baseline for model verification, which is the on-demand analysis of forecast against observation data for specific sites, periods, and element. They also measure the impact of their proprietary forecasting systems against the baseline. Here is how they apply the checks to forecasting systems:

 

Model Output Statistics (MOS): Forecasts are archived in the Verification Database, which is an archive containing raw data for ECMWF, MOS, observations, maritime, and road data. These are used for:

  • Daily monitoring of forecast quality for the weather room
  • Verification for specific sites, periods, and elements on (customer) demand
  • External reports on monthly client-specific forecast KPIs
  • Internal reports on monthly forecast KPIs for the board, which forms the quality retrospectives
  • MOS statistics itself are also used, for example the value of any weather parameter averaged over all stations should not change much when a new MOS forecast is ready

 

Nautical MeteoBase (NMB): Forecasts for grid points near reporting buoys or platforms are archived in the Verification Database for:

  • Daily verification for a subset of buoys and platforms in the North Sea
  • Verification for specific sites, periods, and elements on (customer) demand
  • Monthly client-specific verification of quality-checked offshore observations

 

Scalable Downsizing (ScaDo): ScaDo is a forecasting system for areas with limited observation data and, therefore, there are few verification options. To compensate, the weather experts apply an "out of sample" verification process called ScoreCard during the development of the forecasting method. Once ScaDo is operational, they apply a similar method to a small set of weather stations.

 

Road & Route models: The forecasts are archived in the Verification Database for:

  • Verification for specific sites, periods, and elements on (customer) demand
  • Client-specific road weather verification
  • Quality reporting for route based forecasts using the downscaling technique

 

Why experts are essential for quality controls

 

Experienced meteorological experts play a central role in enhancing and making the best use of the data behind weather forecasts. Because meteorologists make improvements in the forecast systems, quality controls are used to validate whether manual changes were correct. An important part of their role is also creating custom reports for both internal and customer verification; essentially showing the accuracy level of the forecast.

The verification system used by the experts is well defined and transparent, providing context around quality and performance. But not every weather company offers this level of detail. The learnings are applied by the experts to raise the bar on forecast-specific and generic value parameters to help you get the accuracy levels you need to make informed decisions.

 

Sign up for the weekly How It’s Made Update. Stay informed and get weekly updates, delivered straight to your inbox.
 

Sign Up Now

 

 

 

When processing over 20.000 weather stations on the globe with hourly updates of the observation and the forecast, and adjusting all these forecasts with fresh data from several weather models up to 360h, operating a timely and well-designed automated quality control is a must.

Wim van den Berg

Senior Meteorological Consultant
MeteoGroup