A basic of science

By Michael Hawkins

I often find myself reminded of a post I made on just the third day in the life of my blog. It was about a media report on a recent study that said a certain pesticide found in anti-bacterial soaps may actually contribute to the development of antibiotic resistance by bacteria. It was a fine study, but it was far from conclusive. (The news article wasn’t so cautious in its assertions.) Perhaps it would be best if people only used regular old soap, what with that not really qualifying as a real sacrifice, but as for the science, I was far from ready to say that that pesticide was a contributor to antibiotic resistance among bacteria in any significant way in the given environment.

And the reason is quite simple: science does not rely upon individual studies. Of course, we may be able to point back to the results from one lab or one group of researchers as published in a single study as the linchpin that opened up a whole new branch of study. But that doesn’t mean we believe that paper as being conclusive on its own. It only works when we have a body of evidence. In most cases that means a number of studies looking at the same or a similar problem and coming to the same or very similar conclusions. For a single paper that proves itself a linchpin, that means we need a number of other studies which use its findings as their basis. For instance, green fluorescent protein, or GFP, was shown to work as a marker of gene expression in a pretty definitive study. It has about a bajillion (rough estimate) other studies on it, but no one needed to reproduce the study which won one research team the Nobel Prize in chemistry. But people did use that study as a basis for about a gagillion (rough estimate again) studies. If the original study was wrong or faked or otherwise limited, we would be well aware of that by now because of all those subsequent studies. That is one way to compose a body of evidence.

To put this another way, take the studies on intercessory prayer and its efficacy. We have some that show positive results. “Look, God is here to help!” But then we have others that show negative results. “Oh, no! God must be angry!” And then we have a whole bunch which shows a null result. “Uh…God must be indifferent.” So how do we interpret these results?

Remember, we need to be looking at the evidence as a body. As one of those intolerant, bigoted, hate-filled evilutionist atheists, I would find it humorous, at least on one level, if prayer gave negative health results. But I don’t get to have that laugh. Instead, I have to conclude that prayer has no detectable effect on health. None of the studies are conclusive; they suffer from bias, or are statistically insignificant in either direction, or just show a blatant null result. The most likely conclusion is that prayer does nothing. No study has convinced me otherwise, and most of the studies have shown prayer to be inconsequential to the well being of people anyway.

What I hope this post enables readers to do is recognize a fundamental aspect of how science works so that next time they see a study which concludes a link between this or that, they know what to think. That doesn’t mean it is okay to just dismiss a non-bias confirming study (i.e., a study that doesn’t give a result one likes). It just means that it is always necessary to look at the entire body of evidence before drawing a conclusion.