Keeping score: in search of an accurate Environmental Sustainability Index

On top of all this, what might be called 'timing issues' are neglected. The population indicator comprises 'fertility rate' and 'change in population between 2000 and 2050'. The logic is that high population is an indicator of environmental damage. But if they believe this, then these indicators need to be weighted with another which looks at current population (density). Using their current measure, a country with low population but high population growth will score much worse than an already densely populated country with a static population. Yet the latter country could easily have far greater population pressure on the environment. This indicator category is the only one which measures future trends – and this bias is against poor countries. A better indicator might have been current population per acre of arable land.

Then there are the methodological inconsistencies. Three variables measuring the effects of one country on global systems use the measure of impact per capita. However, the CO₂ emissions variable is measured in terms of 'tonnes of carbon times tonnes of carbon per capita' – apparently doing this 'reflects two ways to measure global responsibility'. Yet this implies that simple 'tonnes of carbon' is a way of comparing countries' responsibility. If this were true, then the implication is that all countries should emit equal amounts of carbon – Luxembourg the same as China, Chad the same as the USA. The effect of doing this is to have an indicator of emissions squared divided by population – which creates wild extremes between countries, and has the effect of reducing the score for heavily polluting countries. The ESI uses a methodology for wildly skewed indices which in this case makes higher carbon per capita countries score better than if a simple carbon per capita figure were used.

In addition, inappropriate units are occasionally used in the index. For example, one variable is Forest Stewardship Council-accredited forest area as a percentage of total forest area. This penalises countries with large tracts of forest untouched by forestry. A better indicator would be FSC production as a percentage of total wood production – a better measure of the relative unsustainability of a country's forestry practices.

Finally, there is a bizarre bias against countries from whom no data at all on certain subjects has been collected. The ESI team uses a methodology to deal with nonreporting countries, which we feel can heavily penalise those countries. For example, there are three variables in the 'Urban Air Pollution' indicators. There is no data for these three variables for Haiti, yet Haiti is put in 98th place in this indicator. Similarly, the nuclear waste indicator has data missing for 77 countries and it appears that some, if not many, of these 77 countries have not been given the obvious '0' for nuclear waste (as they have no nuclear facilities), but some oddly extrapolated, larger value.

Summarising the mistakes

The ESI team says "unlike many efforts to think about indicators of 'sustainable development', we have focused on environmental sustainability, which is a more narrow formulation. This choice was made deliberately, based on a conclusion that one reason efforts to measure sustainability fail is that they seek to fold too many disparate phenomena under the same conceptual umbrella. While we accept the premise that politics, economics and social values are important factors worthy of being sustained, we do not think there is a sufficient scientific, empirical or political basis for constructing metrics that combine all of them along with the environment. Moreover, the environment often gets overshadowed in 'triple bottom line' analyses and other sweeping sustainability efforts".

While this may be true, the fact is that the ESI, with this survey, falls into its own trap. The inclusion of social and economic indicators allows it, conveniently, to conclude that the world's richest, highest consumption, highest-impact nations are the true protectors of the environment – and that the poor need to follow their lead.

How it should be done?

Some of the data in the ESI can be used to create a far better measure of 'environmental sustainability' – and this we have tried to do here. Our approach has been first to cull all the non-environmental indicators, then all the ones with methodological problems. This left us with indicators in seven areas. Four relate to how a country uses its own environmental resources – with effects on land, air, water and biodiversity. The other three relate to a country's use of global environmental resources – the categories being pollution to land, pollution to air, and resources extracted.

We used the same methodology and data as the ESI to calculate rankings for these (sometimes different) indicators. Using the ESI's methodology, values for the variables for each indicator were averaged to get indicator scores; so the revised index totals are the average of the seven indicators we used. The one change made was to calculate CO2 emission scores based on per capita emissions. We believe this simplified index (comprising seven indicators and 15 variables, rather than 22 indicators and 67 variables) captures true 'environmental sustainability' much better than the original.

The real story

When these more genuine indicators are factored in, and some of the more bogus ones removed, the final results tell a very different story about which countries are really environmentally sustainable – and which aren't.

One of the most obvious examples is the very unsustainable US which, under these new calculations plunges from 11th in the rankings to 112th. At the other end of the scale, the Central African Republic, Bolivia and Mongolia are elevated to the top three. This reflects the good environmental conditions in their own country, and the small effect their development has on global ecosystems.

In all, richer countries do worse – for although they often have good environmental conditions at home, and manage to protect what's left of their biodiversity, they have a large negative impact on global ecosystems. Some countries, of course, score badly whichever way you look at it – the bottom three are South Korea, Kuwait and the Lebanon.

What conclusions should we draw from this? Simply, if we are going to label nations 'good' or 'bad' in environmental terms, we must get our measurements right. Studies like the ESI, based on misleading data, which fail to take into account the true environmental costs that rich countries impose on the world, are designed to make dirty nations look clean.

When the 'Global Leaders for Tomorrow' next sit down at their calculators, their results should tell a very different story.