How it's made: Why You Should Care About Quality Control & Data Management of Weather Data
We’ve arrived at Category Four in the How It’s Made series. This section focuses on everything related to quality control and data management; it explores why quality improvement and data management is so important.
Weather forecasting is work in progress. As we discover new weather phenomena , implement new technologies, and improve data models, we can work to make your forecast even better.
Make sure you don’t miss out on the rest of the series. Sign up for the weekly How It’s Made Update. Stay informed and get weekly updates, delivered straight to your inbox.
Weather forecasting is not something you have; it’s something you do. Just like playing football and learning to drive a Formula 1 car, you have to work on your skills to become better and more successful in the future. This is important if you’re Lionel Messi or Max Verstappen, but also if you’re a professional meteorologist.
Every day, new weather data comes in that has to be checked for accuracy, completeness, and irregularities. New and better calculation models not only improve weather forecasting, but also improve the accuracy of historical data, which is equally as important to make predictions. Clearly, quality control and data management are worth the time. But what do the weather experts check? And how do they ensure it’s accurate?
"Our clients make decisions based on our data. These decisions can affect the safety of people and involve high costs. At MeteoGroup, we are always aware of the high responsibility that comes with our crucial role in the processes of clients. Providing highest accuracy for those weather situations where our clients and their downstream activities are most vulnerable is key for us. When we measure data accuracy and forecast quality we take a strict user perspective. We apply scores which are scientifically sound and still intuitively understandable."
So many reasons to be accurate
Let’s start by answering a very important question: why are the continual quality and data management improvements so important? First of all, the difference between somewhat accurate data and highly accurate data can make or break a business. For example, weather routing gets ships from A to B on the optimal route.
On the flip side, inaccurate forecasts cost businesses. False alarms in offshore operations can unnecessarily add days to projects, increase costs, whilst on the other side under-forecasted winds or waves put the lives and safety of personnel at risk. No matter what industry companies work in; they rely on weather experts to provide them with accurate information on the weather so they can make better decisions. This is why transparency and accuracy are essential parts of every forecast, every single day.
Observational data is gathered in several ways, such as through weather stations, radars, satellites, and lightning detection. Although “observing the weather” sounds rather simplistic, it really isn’t. Instruments, coverage, standardization; they can all be improved to provide better observations. When it comes to weather stations, for example, meteorologists continuously monitor the arriving data streams for measurement and transmission errors while standardizing the data for universal use. They also apply specific quality-checks for historical observation data to ensure consistent time series. At the same time, the scientists keep on improving the algorithms that clean precipitation radar data, by removing false echo signals. As for satellite data, images are checked for completeness to prevent black spots in presentation.
Meteorologists draw data from multiple weather models to improve the accuracy of forecast systems like MOS, nautical models or road models. For each model, they do baseline measurements to verify its accuracy for specific sites, periods, and elements on demand. At the same time, they monitor their performance at the crucial decision points of clients. Additionally, the meteorologists measure the impact of the proprietary forecasting systems against the baseline, to make sure they stay top of the class.
In addition to all of these quality checks, meteorologists also draw up monthly reports on client-specific forecast KPIs, so they can be compared to the months before. They need this information to keep track of their improvements for the client. To top it off, they also draw up monthly reports on forecast KPIs for internal use, as a quality retrospective.
Data Management is responsible for ingesting, processing and storing all the incoming data; both sourced and generated. Once it has been processed to the high standards required, the data is shared with various departments and forms the backbone of products.
When it comes to the quality of data, weather models and forecasting systems that you invest in, you should spare no expense. The market pressures demands highly accurate weather data and strategic support, so that’s what they should get. Each new day brings more information that helps the weather experts to improve, which is why they will never be done investing in quality control. Call it obsession, call it passion; it doesn’t matter. At the end of the day, it’s the result that counts.
Sign up for the weekly How It’s Made Update. Stay informed and get weekly updates, delivered straight to your inbox.
Our clients make decisions based on our data. These decisions can affect the safety of people and involve high costs. At MeteoGroup, we are always aware of the high responsibility that comes with our crucial role in the processes of clients. Providing highest accuracy for those weather situations where our clients and their downstream activities are most vulnerable is key for us. When we measure data accuracy and forecast quality we take a strict user perspective. We apply scores which are scientifically sound and still intuitively understandable.