The complexity problem

The story of civilisation is in large part the story of increasing scale and complexity in human affairs. We are now reaching limits. Already civilised life is now so complex that further complexity threatens overall collapse. It is this stark reality that underpins many of the current headline issues.

The news is currently full of big changes, mostly containing some threatening aspect. For example, the IPCC recently realised its latest report on climate change confirming the size of the global warming problem and human culpability. The world economy staggers on with most of the problems of the financial crisis still there and most national economies still in trouble. Big financial corporations have been caught time and again breaking the law: JP Morgan recently paid out over $5 billion dollars for various crimes, but this is just the tip of the iceberg of financial shenanigans.

The US, and the global system it still runs, are in accelerating decline, with one recent disaster being the NSA spying fiasco (it seems the US's main intelligence agency has been operating pretty much out of control for years). Indeed, more and more stories appear about the growing role of digital technology in human affairs, from killer drones to the way it is shaping the plastic brains of the young.

All these matters reflect the growing problem that is the relationship between scale and complexity. Our modern world is now far beyond the understanding of all but a few, but we still live affected on a daily basis by what is happening. The roots of this problem go way back to when modern humans first evolved on earth.

In terms of scale, we have gone from hunter-gatherer band to agricultural hamlet to city to empire to nation state to global civilisation over 10,000 years. Hunter-gathers roamed far and wide but in small numbers, and life was comparatively simple (although it was already relations between humans that was driving the changes in our brains). Once we concentrated into communities they grew spatially larger and more numerous in terms of population. We went from small villages of hundreds to cities of thousands to empires and then nation-states of millions to a global system currently of over seven billion people. We developed social hierarchy, division of labour and new technology (like writing) to deal with the growing complexity caused by so many people.

In terms of spatial organisation and population we have pretty much reached our limits here on earth, and to expand further we'd need to head off into outer space. In the 1950s and 1960s this looked possible, even likely, but the spaceships that put men on the moon were the most complex machines ever built up till that time, and it was mostly dumb luck that enabled the manned space program to do as well as it did. Right now, except for unmanned research missions, manned space travel looks too difficult and we stay marooned on Earth.

Complexity is in part a function of scale as greater scale demands more complex systems to maintain control over distance. The key is speed of communications. A village could be organised by people talking to each other, a city by written edicts posted on walls or read aloud in main squares, an empire by reliable road and sea transport and coded communiques, and modern nation-states by telecommunications, these days digital in form.

The basic problem is that we have been used to creating more complexity to deal with failing control, but now the very level of complexity generates its own problems. Our institutions and their operations are so large, their interactions so speedy, we humans are increasingly left behind.

This explosion in complexity manifests in our daily lives as a sense of always being rushed for time, of having too many choices, of everything we do being over-complicated. Increasingly, people seek a 'sea change' or 'tree change' and slow food to recapture a sense of real value.

Perhaps the best example of this trend to over complexity is finance, which was originally devised to help control trade of goods and services. Originally trade was the dog, finance the tail, but now a huge tail wags a much smaller dog. Finance has had an international aspect since the first states and empires, it became really powerful in the nineteenth century in the hands of men like the Rothschilds, and became genuinely global in scale after the catastrophes off World Wars One and Two, led by the mostly American big banks.

Post-war reconstruction gave a new role to big finance, but it really took off due to two basic factors: Nixon ended the role of gold in 1971 and in the 1980s digital technology made 24 hour trading at a global level possible. Now the money that shuffles at near light speed around the world dwarfs the real world economy of goods and services, but the global finance system was inherently fragile. The global financial crisis that began in 2008 and has never really ended showed just how fragile.

It is doubtful that anyone actually understands how the global finance system works these days, let alone how to stop it from convulsing all the time. Financiers increasingly rely on software, or quants, to trade, but these programs just add to the complexity. Currently trading programs can complete 100,000 transactions in a tenth of a second, even as they seek to mislead other trading programs. So-called regulators have just about given up, following behind the latest scams and tut-tutting.