In defence of the Bureau of Meteorology's forecasting of tropical cyclones

More than three weeks ago, Jennifer Marohasy delivered a harsh assessment of the performance of the Australian Bureau of Meteorology in its forecasting and reportage of Tropical Cyclone Marcia which had crossed Queensland's Capricornia Coast days before (OLO, 23 February). The cyclone's intensity had been over-forecast, Marohasy claimed, and the communication of the data produced by the event had been mishandled. The Bureau had predicted a Category 5 storm on landfall; the writer claimed that its peak wind speeds made it at most a Category 3 cyclone on crossing the coast near Yeppoon.

Marohasy finished with a flourish by linking the Bureau's forecasting behaviour with the thoughts of "those ever animated by the idea of catastrophic global warming, a monster category five cyclone has roared ashore for what would otherwise have been a disappointingly quiet cyclone season."

There was emotiveness and seeming sarcasm and slant in these words. Evidently Marcia had saved the Bureau's cyclone season by creating interest at last: a good season is a big season, not one in which little happens. More importantly Marohasy implied that the Bureau's performance was incompetent and perhaps professionally dishonest, and partisanship in the Bureau's position on climate change was also suggested. A statement seemed to have been made that the organisation is biased in an important debate.

Marohasy's piece was followed by a lengthy thread of responses, 41 comments being posted to OLO within about a week. Strong views were expressed, the Bureau being defended by some while others supported Marohasy's criticisms on both the forecasting and the reporting of what had actually happened. There were roughly equal numbers for and against the positions the author had taken, and much vitriol and sarcasm was in evidence. The exchanges were not particularly edifying, as the editor (Graham Young) noted, but they pointed to some characteristics of modern debate about the forecasting of weather.

We seem to have a default position of mistrusting the Bureau. Perhaps this is a feature of all societies; it certainly appears to be true of ours. We expect the Bureau to get all its forecasts right, and many condemn it when it errs. The 'teaser' at the start of Marohasy's article, presumably written by the editor, was a case in point: the Bureau had caused "distress" to "millions of people" and should be investigated. Later, in a post within the comments thread, the editor suggested that the "BOM has to apologise, withdraw and probably terminate someone's employment". For that to be necessary, presumably, incompetence or dishonesty must be present and clearly demonstrated.

And the editor referred to some of those who disagreed with Marohasy as "trolls", while the posters on the other side of the argument were not so impugned. Yet as far as I can see there was rudeness on both sides. The Bureau was treated badly in the thread, with insinuations or accusations of spin and lying, as was Marohasy.

Is some of the criticism of the Bureau not going too far? The comment about millions of people being distressed is surely an exaggeration: the area for which cyclonic wind speeds were forecast in Marcia's case probably contains only about a half a million people. And was the Bureau's forecasting so much in error? Certainly, winds of Category 5 strength appear not to have been recorded at the few stations over which the system passed, but that ignores the likelihood that greater speeds than were measured were experienced at higher, more exposed sites. Measuring sites cannot be assumed to capture the most extreme conditions produced by a particular weather event.

There was no sense in Marohasy's piece or in the editor's comments that the Bureau would receive any benefit of the doubt. Nor was there any recognition of the fact that the Bureau uses internationally standard methods of forecasting which are known to like organisations the world over. Moreover its staff are trained according to methods that are accepted world-wide. Science and its applications these days are thoroughly globalised. What happened with Marcia may have been a case of critics 'situating the appreciation', conclusions being drawn to fit preconceived positions. One gets the impression that the Bureau's incompetence was taken as a given and this informed some analyses. Certainly this seemed to be so with the comments of some who weighed into the commentary thread.

The truth of the matter, perhaps, was that the Bureau had correctly forecast an unusually rapid intensification of Marcia's cyclonic characteristics (including wind speeds) as the system approached the coast. Marcia's track, too, was predicted with considerable accuracy. The strength of the winds as the system made landfall and as it tracked south thereafter was somewhat over-forecast, but little real harm was done thereby. I am unaware from the media accounts as the storm approached or after its passage that there was "distress" caused to many by the forecasting (though there would have been plenty of distress felt by those whose houses were actually being damaged or destroyed). Concern, yes, including on the part of people not themselves at risk but worrying about friends and relatives who might have been.

Here, perhaps, it should be noted that the forecasting associated with tropical cyclones Nathan and Olwyn over the past week seems to have been quite accurate with regard to wind strengths and storm paths. Nathan, though, was eventually recorded as a Category 2 event rather than Category 3 as was forecast for a time. Here it should be said that forecasting cyclone paths is complex, as is estimating how low central pressures will go and how much rain will fall in particular areas. The science, worldwide, is still developing on these matters. Moreover forecasting agencies are encouraged to report their best estimates as to what will happen, not to remain quiet in the face of uncertainty about it.

As a community we probably have unrealistic expectations of the Bureau. It routinely attracts criticism for alleged and actual errors of prediction.

Interestingly, few complained when Russia's annexation of Crimea a year ago apparently was not forecast by the West's intelligence agencies, and there was precious little advance warning in 2007-08 of the Global Financial Crisis by the world's economic forecasters in the banks, in governments and amongst financial journalists. Predicting the future is a fraught activity, and getting it wrong on occasions is inevitable. Some degree of error needs to be accepted given the complexities often involved. And it should be said that in the case of weather emergencies an element of over-forecasting is preferable to a similar element of under-forecasting, though obviously neither is ideal and extreme errors are very much to be avoided.

What needs to be asked is whether evidence of an impending event was competently read in the lead-up to it, how serious the degree of error was and what consequences flowed. And the limitations of current scientific knowledge should be appreciated. None of that, incidentally, means that legitimate criticism is impossible.

The Bureau's severe weather forecasters use the standard, tested and accepted tools of their predictive trades that are employed around the globe. I find it difficult to believe that as a general rule they do a poorer job than is done by their peers in other national weather forecasting organisations, but it would be fascinating to see an objective comparative study on this question. Equally I find it hard to believe that they fudge their results (and the same goes for the vexed issue of the homogenisation of gauge records on temperature, another matter on which the Bureau has been criticised by Marohasy and others). It would be too easy to be found out.

Until there is credible evidence that Bureau staff are not playing straight, and I have yet to see it, they should be given the benefit of the doubt.