Keeping score: in search of an accurate Environmental Sustainability Index

• Duplication

Some of the capacity indicators duplicate measurements already used elsewhere in the ESI. For example, there is a 'price of premium gasoline' variable within the 'capacity' section – but there is already an indicator of the end result – CO₂ emissions, and air pollutants. The price of gasoline has a greater effect on the overall ESI than CO₂ emissions.

• Poor proxies for positive action

Some indicators are not directly related to environmental sustainability at all. An example is the 'number of scientists per head of population'. The effect of science on environmental sustainability is, of course, dependent on how it is deployed. Having lots of scientists around is not much good for environmental sustainability if their efforts result in nuclear wastes, endocrine-disrupting chemicals and ozone depleters, even if some short-term social or economic gains are made. Again, the potential for capacity to be used in a negative way is not recognised.

• Not enough on global indicators

The indicator set is very focused on environmental effects or institutions within countries. However, many countries escape environmental constraints in their own country by drawing on environmental capital either from other countries or from global commons. The biggest overexploitations of nature's carrying capacity are occurring at a global level – fish stocks, climate change, ozonelayer depletion. There are variables in some indicators which measure global impact – but these amount to just over one indicator out of 22 – around 5 per cent of the index.

• The weightings

The ESI team decided – after exploring various angles – not to assign different weightings to any of the indicators. However, by getting to 22 'equally weighted' indicators, substantial weighting has already been done – in choosing the 22 and what they comprise. The category of 'Capacity' has seven indicators, for example, while the category 'Global Stewardship' has just two. There is a whole indicator measuring countries' economies' carbon intensity – yet the end result this affects, carbon dioxide emissions, the cause of worldwide climate change, gets only two fifths of an indicator.

Moreover, the ESI team did in fact conduct a weighting survey – whose results put five of the seven capacity indicators in the bottom six of the 22 indicators, in order of importance. There is an overwhelmingly disproportionate bias towards the indicators of 'Capacity', which is a major cause of the index's high scores for rich countries.

• Individual problems

There is also a wide range of other problems with many of the specific indicators themselves. For a start, indicators are used in one country when the problem may be caused by another: incidences of acid rain being an obvious example. On top of this, major problems are omitted, leading to bias against particular countries.

The environmental health indicators, for example, have variables of 'child death rate from respiratory diseases' and 'death rate from intestinal infectious diseases'. These could be caused by a number of factors, not necessarily due to environmental problems (lack of public health service, for example – a socioeconomic issue), and/or by factors already covered (such as access to safe drinking water). In only including infectious and respiratory diseases, major categories of illness are missed out – such as environmental ill health resulting from Chernobyl, Bhopal etc.