Flawed forecasting

For a cautionary tale about how not to take forecasts, any forecasts, seriously we need look no further than the strident forecasts of American criminologists in the 1980s.

As recounted in the book Freakonomics (Allen Lane, 2005) by economists Steven D Levitt and Stephen J Dubner, when the US crime rate soared in the 1980s various criminologists forecast that crime rates would rise further in the 1990s.

As the causes of crime were known in broad terms even 20 years ago you would have expected them to at least be in the right ball park in their forecasts. Instead crime rates fell sharply, across the country. Anyone interested in further details should read Levitt and Dubner’s entertaining book.

Forecasting disasters of that magnitude are quite common. They are legion in business and barely worth mentioning in economics or social analysis. Weather forecasting is somewhat better perhaps because meteorologists have had a lot of practice at it, and have the help of satellites and super computers. But even so, I recall being told by one senior meteorologist some years ago, that simply guessing that tomorrow’s weather would be the same as today’s gave nearly as good results as that of any of the meteorological bureaus.

The involvement or otherwise of experts does not make much difference. See the 2007 paper Global Warming: Forecasts by Scientists versus Scientific Forecasts (PDF 749KB) by J. Scott Armstrong and Kesten Green. For those interested in the subject of forecasting this site (PDF 96KB) on public policy forecasting is interesting.

But in recent years a group of vocal scientists and even more vocal activists have been insisting that a set of forecasts produced by the Intergovernmental Panel on Climate Change, linking industrial emissions to temperature increases, are not like the legion of forecasts that fail. In fact, this group says, they can be relied on to such an extent that we must spend billions in efforts to reduce the amount of carbon dioxide being pumped into the air. Further, they are so sure of the results that anyone who dares to challenge the forecasts is accused of being ignorant or working to some sort of political agenda.

What reason can there be for such extraordinary confidence? About the only way we can make any judgment about these forecasts is to look at the track record of the forecasts to date. If we look at the recent results of from Hadley (Hadley is used throughout) without benefit of extensive training in climatology it looks as if the annual mean temperatures are coming off a peak. Yes there is a distinct increase in temperatures in previous decades, but one of the rules of forecasting (see the material cited above) is that achievements must be measured against data unknown at the time of the forecast. On that measure the IPCC forecasts do not fare well, as far as we can tell to date.

The set of forecasts issued in 2001 by the IPCC, forecast a top rage increase of 0.6 degrees a decade and a bottom range of 0.1 degrees. In the noticeable increase between 1975 and around the end of the century, the rate of increase may have briefly touched 0.3C a decade. In other words the IPCC issued its forecast expecting that temperatures increases evident in the previous 25 years would at least be the same, or accelerate markedly. Instead, just like the forecasts of the American criminologists, the ink on the IPCC report was barely dry before the actual system started doing the exact opposite - again, as far as we can tell. The forecast was issued just after an apparent peak in annual temperatures to judge from the five-year running average, followed by a gentle decline.

The 2007 IPCC report (PDF 4.1MB) said much the same things as the 2001 report except for a top range increase to around 0.45C a decade. Temperatures have since fallen further at all centres that measure temperature (with the possible exception of the Goddard Institute of Space Studies). Global warming proponents can get around this to a certain extent by pointing out that the forecasts run from 1990, and if taken from that year temperatures can still be said to have increased, albeit not by as much as the mid-range of the forecasts.

Another response is to point out that the last few years still count as among the warmest on record. Perhaps they are - there have been suggestions that the 1930s were hotter - but its beside the point. Warming was supposed to have accelerated. Instead, as far as we can tell to date, temperatures went in the opposite direction.

In short, to speak convincingly of tipping points, one has to point to a forecasting track record, and it isn't there. For that matter the climate models at the centre of all the fuss have yet to demonstrate any real success in modelling known climate changes outside the past 100 years or so, such as the Medieval Warm Period or the Little Ice Age or other major periods or earth’s history. (See the New Scientist cover story of June 21, 2008, “Once the South Pole was green”.)

What of all this talk about melting glaciers? What about the Wilkins Ice Shelf which is supposed to have collapsed recently, after more than a century of proud existence? In fact, the ice shelf is repeatedly reported as having broken up, the first time back in the early 1990s. Although some glaciers have been melting faster than scientists have previous expected, there is a lot of argument about what the increased melting is due to. Also, as there was an increase in temperatures up to the end of the century one would expect some continued melting.