Science death spiral

Having brought mankind so far, has traditional science finally outlived its usefulness? Many seem to think so, finding the rules of the scientific method—the strict guidelines a researcher must follow to actually practice science—far too restrictive and cumbersome. The requirement that evidence be empirical, which is to say, actual measurements of nature itself, is found too burdensome to new age scientists. They prefer clean, clinical computer models to messy, often uncooperative nature. Over reliance on models, misapplication of statistical methods, and lack of repeatability are the hallmarks of the new pseudoscience that is replacing the traditional practice of science, real science. As one critic recently wrote: “The problem with ­science is that so much of it simply isn’t.” Has science entered a death spiral, as indifferent, inept scientists raise up new generations of even poorer researchers? The facts look grim.

There has been a flood of recent articles addressing the failure of modern science in both scientific journals and online. The respected journal Nature posted a new article entitled “Online debate erupts to ask: is science broken?,” addressing just this subject. In it, results from a panel discussion, held at the University College London, were reported. These included claims that some dubious laboratory practices, such as tweaking statistical analyses to make results seem significant, are widespread.

From psychology and medicine to the physical sciences, alarms are being raised that science in its evolving modern form is failing. It is as if the rules, which have served science so well for centuries, have become optional. Requirements for judging good science from bad have become flexible, expanding to embrace sloppy research and even outright fakery. I have written about the scientific method and the philosophy of science before (see “Cherry Picking, Black Swans and Falsifiability”). I have stressed how theories must be testable, meaning they must make predictions that can be either confirmed or refuted by actual measurement. Beyond that, these measurements must be repeatable by others. In other words, an experiment can not just work for the originator of a theory. That is a good place to start investigating modern science's failures.

Writing on the First Things blog, William A. Wilson has penned a scathing critique of published research in the field of psychology. In “Scientific Regress,” Wilson reveals modern psychological research to be a house of cards, filled with unreproducible results:

The problem with ­science is that so much of it simply isn’t. Last summer, the Open Science Collaboration announced that it had tried to replicate one hundred published psychology experiments sampled from three of the most prestigious journals in the field. Scientific claims rest on the idea that experiments repeated under nearly identical conditions ought to yield approximately the same results, but until very recently, very few had bothered to check in a systematic way whether this was actually the case. The OSC was the biggest attempt yet to check a field’s results, and the most shocking. In many cases, they had used original experimental materials, and sometimes even performed the experiments under the guidance of the original researchers. Of the studies that had originally reported positive results, an astonishing 65 percent failed to show statistical significance on replication, and many of the remainder showed greatly reduced effect sizes.

According to Wilson the problem is not limited to psychology. An investigation into the pharmaceutical industry at Bayer, focused on cancer biology research, researchers found that more than 75% of cases the published data did not match up with in-house attempts to replicate. As Wilson points out, such conformation failure can be because the reproduction was in someway flawed or the original finding was false. Moreover, the failure of an experiment may well be because of failure on the part of the researchers to understand the subtleties of Bayesian statistics. “A tremendous amount depends on the proportion of possible hypotheses which turn out to be true, and on the accuracy with which an experiment can discern truth from falsehood,” he writes.

John Ioannidis, a professor at Stanford University’s School of Medicine, recently published a paper with the shocking title “Why Most Published Research Findings Are False.” In it, he details why the validity of an experiment depends on the proportion of possible hypotheses which turn out to be true, and on the accuracy with which an experiment can discern truth from falsehood. Ioannidis shows that for a wide variety of scientific settings and fields, the values of these two parameters are not at all favorable. In many cases, approaching even 50% true positives requires unimaginable accuracy. Ioannidis's 2005 paper has become the most downloaded technical paper from the journal PLoS Medicine.

But it is not just the medical and social sciences that are at risk. Physics, that most concrete of the physical sciences, is supposedly based on measurable experiments: dropping objects from a tower, the bending of light by an intervening galaxy, Millikan and his oil drops, etc. Surely physics lies at the other end of the scientific spectrum from sciences like psychology and sociology where interpretation plays a much larger part. According to Wilson that would be a false assumption.

[M]easured any way you like—volume of papers, number of working researchers, total amount of funding—deductive, theory-building physics in the mold of Newton and Lagrange, Maxwell and Einstein, is a tiny fraction of modern science as a whole. In fact, it also makes up a tiny fraction of modern physics. Far more common is the delicate and subtle art of scouring inconceivably vast volumes of noise with advanced software and mathematical tools in search of the faintest signal of some hypothesized but never before observed phenomenon, whether an astrophysical event or the decay of a subatomic particle. This sort of work is difficult and beautiful in its own way, but it is not at all self-evident in the manner of a falling apple or an elliptical planetary orbit, and it is very sensitive to the same sorts of accidental contamination, deliberate fraud, and unconscious bias as the medical and social-scientific studies we have discussed. Two of the most vaunted physics results of the past few years—the announced discovery of both cosmic inflation and gravitational waves at the BICEP2 experiment in Antarctica, and the supposed discovery of superluminal neutrinos at the Swiss-Italian border—have now been retracted, with far less fanfare than when they were first published.

So even the hard sciences are susceptible to irreproducibility. In “Why Science Is Broken, and How To Fix It,” scientist William M Briggs weighed in on the problem. “There’s been a spate of lamentations that science is broken,” he writes. “I am a credentialed, working scientist, and I’m here to tell you that, with some exceptions, these cris de coeur are right. Science is a mess.”

Decrying “nitwit, avaricious, power-hungry politicians” who promote the “science is settled!” mantra, Briggs points the finger at bad politics, bad money, and bad philosophy. He complains that too many indifferent students have swollen the ranks of researchers and too much (not too little!) money is driving mediocre scientists to do mediocre research which gets published in content hungry journals. Climate science gets particular mention.

First problem is over-certainty. Science works where it is forced to confront reality. This is why meteorological forecasts have improved so greatly, and why climatological forecasts haven’t. Weathermen have to perform. Climatologists have to please politicians and funding agencies. Which group is cockier?

That climatology, and some other fields, succeed when they shouldn’t is a philosophical problem. It used to be understood that scientific theories should match reality. But that is a philosophical and not a scientific proposition. Indeed, all the principles that define science aren’t themselves scientific.

He then takes solid aim at the overuse, indeed the misuse of statistics as a crutch for sloppy research.

The second problem is ritual. Many, many fields rely on probability models for their livelihood (the bulk of sociology, for example). But almost none of the people using these models gets them right. Everybody knows the old saw “correlation is not causation”, but just as the guy in the pew doesn’t think the sermon is meant for him, most scientists don’t think they’ve fallen prey to the fallacy of confusing correlation and cause, though many of them (particularly in the soft sciences) have.

I myself have seen the rampant spread of statistics at all levels of science. In the past I have served as a judge at the state science fair, held at a local university where I have taught on occasion. An astounding number of young researchers held up a cooperative p-value as unassailable proof that their experimental results were spot on. When I commented on this to one of the other judges, a biologist, I received an almost indignant reply that the p-value was the “statistical gold standard” and that they were ingraining this in all their students. The problem here is not doing the calculations—there are many packages that will do that—it is in the construction of the experiment, the validity of the assumptions that frame the null hypotheses, and the interpretation of the results.

Clear scientific thinking has been supplanted by a magical statistical blessing. Stating “The hunt for p-values less than 0.05 has left many of science’s roadways riddled with potholes,” Hilda Bastian offers up “Tips For Avoiding P-Value Potholes,” though whether others will heed her warnings remains to be seen. So wide spread is the abuse of the p-value that the editors of Basic and Applied Social Psychology (BASP) announced that the journal would no longer publish papers containing p-values because the statistics were too often used to support lower-quality research.

Yet another article proclaims “the replication crisis in science has just begun.” Its summary states: “After a decade of slow growth beneath public view, the replication crisis in science begins breaking into public view. First psychology and biomedical studies, now spreading to many other fields — overturning what we were told is settled science, the foundations of our personal behavior and public policy.”

The author focuses on the core problem of reproducibility, because results that cannot be reproduced are useless in science. Posted on the Fabius Maximus blog, it is a treasure trove of links to other article regarding science's malaise. Included is the central cite for monitoring retraction of scientific research, the appropriately title Retraction Watch.

Bringing all this back to climate science (you knew I would) in a stunning new paper, "Climate Modeling Dominates Climate Science," by Patrick J. Michaels and David E. Wojick, the extent of over reliance on climate modeling in climate science has been exposed. The research paper surveyed the entire literature of science for the last ten years, using Google Scholar, looking for modeling. They found that climate change science accounts for fully 55% of the modeling done in all of science. Quoting the article:

In fact the number of climate change articles that include one of the three modeling terms is 97% of those that just include climate change. This is further evidence that modeling completely dominates climate change research.

This shows how fake climate science "research" really is while at the same time tarnishing the reputation of computer modeling, which is a useful tool when applied properly. It's not just GCM, every aspect of climate science has been infected with modeling fever (see “Of Models And Melting Ice Caps”). What's more, modeling is spreading to other fields of inquiry, tempting researchers to invent their own computer realities rather than investigate nasty, inconvenient nature.

To summarize, the following are the factors that are eroding the pillars of science.

• Unreproducible results – through shoddy work, poor experiment design, and statistical ignorance more and more results reported in papers can not be reproduced, making them scientifically useless.


• Corruption by politics – whether through group think or government funding the pressure to conform to politically acceptable results has increased to the point working scientists either submit to consensus or stay quiet.


• Statistical malpractice – through lack of training or sloth, many scientists use statistics as a drunk uses a lamp post, for support, not for illumination.


• Reliance on computer modeling – computer modeling is a wonderful tool when looking for insight but they are not faithful representations of nature itself. When scientists end up studying their models instead of nature they are no longer scientists.


• Misuse of peer review – instead of functioning as academic quality control and an aid to authors, peer review has become the enforcer of consensus thinking and scientific dogma. Instead of helping science advance it ensures conformity.

It should be clear from the above that the standards of science have been greatly degraded in modern times, to the point that many if not most scientific claims are suspect. No where is this more evident than with climate science, a field that has turned peer review into a check for doctrinal orthodoxy that suppresses any contrary thinking or results. Scientists have stopped doing science in the pursuit of publications and government grant money. The herd of mediocre scientists needs to be culled. The only way out of this morass is to remove much of the government funding from fields rife with bad science. The influence of government and UN agencies needs to be greatly reduced—it should be possible to be a bureaucrat or a scientists but never both.

Is science in a death spiral? When I was young I dreamed of becoming a scientist, to search out nature's secrets beholden only to nature and the truth. After many years of study and work I became a scientist, though it was not what I thought it was when I was a child. Most of the images of scientists that inform the public and set their expectations come from TV and the movies. I recall one line from Dean Yeager in Ghostbusters: “You seem to regard science as some kind of dodge... or hustle. Your theories are the worst kind of popular tripe, your methods are sloppy, and your conclusions are highly questionable! You are a poor scientist, Dr. Venkman!” Unfortunately, we seem to have raised up an entire generation of Venkmans, and they are destroying science just when we need it most.

Be safe, enjoy the interglacial and stay skeptical.