Weather forecasts are always wrong, but with more data they could be right. Not only more accurate, but also more specific. How to get the data and how to process it? The Weather Company might have an answer and it is a surprising one.
Weather forecasting is one of the areas of computation that really hasn’t had a big revolution due to the availability of big data. Places that get “good” forecasts are generally in areas where there are extensive meteorological stations – Europe and the USA mainly. The big problem is that such stations are expensive and there is still a lot of weather happening in between them.
The Weather Company, part of IBM, has an audacious plan to get more data and to process it quickly enough to provided hourly forecasts on a much finer grid than before. Global High-Resolution Atmospheric Forecasting System (GRAF) will also manage to forecast local events such as thunderstorms on a 3km, or better, resolution.
The data gathering method is interesting:
This will have any professional meteorologist jumping up in the air and protesting. Real weather stations have carefully constructed instruments installed in specific ways so as to standardize the reading. Taking readings from amateur weather stations is quite another matter. I have a home weather station and the error in pressure, temperature, wind speed and direction are quite large. It is good enough for my consumption and to show trends but I’m not convinced that the data is high enough quality to use in a model of the sort IBM are proposing. The noise could out weigh the advantage of the extra data.
The problem of processing so much data is solved using IBMs Power System “super” computer. This has a number of NVIDIA Tesla V100 GPUs to parallelize the computation:
The importance of accurate local weather forecasts should not be underestimated. Everything from event planning to when to harvest would benefit from such information and IBM will start to make the data available later in the year – for a charge. I wonder how all of the contributors to the data will feel about supplying their data for free to a commercial enterprise.
It’s all an interesting idea and we will have to wait to see if it all works out, but this is indeed the century of big data.
or email your comment to: [email protected]