The press has been reminding us again that the U.S. is losing its international leadership in the sciences (see Jim Holt in the New York Times Magazine, Sunday 12/11/05). And it is true. Many other countries now perform as well as we do in the production of new graduates in the fields of engineering and mathematics and in internationally coveted prizes in the sciences; our advantage in the production of new patents is diminishing. But most worrisome is not that we are losing all the immediate benefits of the scientific advances that would have been ours, but that without an understanding of the rigorous and evidence-based methodology of science, our population confronts many other complex social and political issues with minds steeped in mysticism, gullible to charismatic charlatans, incredulous of the fantastic. And most of these unprepared students grow up to be voters -- and some grow up to be policy-makers whose decisions have enormous impact on our families and our culture. The great scientist Thomas Jefferson would despair.
What parts of the scientific method are our students missing? Sadly, the list is long. Recent events in Kansas and Pennsylvania suggest that we voters and policy makers are not good at recognizing when the accumulation of scientific evidence has tipped a conclusion from theory to fact. But in this post I would like to focus instead at the other end of the evidence spectrum -- where evidence is virtually nonexistent. In that case, how is it that so many Americans are still quite certain about their pet ideas for which there is no valid evidence? I am thinking not only of healing by crystals, of sexual dalliances with aliens, and of fake moon-walks on secret movie studios, but of abstinence-only sex education as an effective way to reduce public health problems, of skyrocketing national debt as a benign and routinely manageable phenomenon, and other extraordinary public policy claims in dire need of extraordinary evidence -- or even of any substantive evidence.
Allow me to illustrate this problem with an example. I have encountered professional and educated persons who firmly believe the following: among people they had observed in their lives, the children of women who gave birth at a later age, say, thirty-five, seemed to grow into sophisticated and well-adjusted adults, but the children of women who gave birth as teenagers seemed to struggle more with behavioral problems and emotional adjustment. They concluded that something about a woman's cumulative life experience can directly and physically alter the eggs within a woman's ovaries, conferring related benefits to the fertilized egg, and thus to the child itself. In fact, the idea is so appealing to them that they have decided outright that this is, in fact, true, and they believe it deeply. They are ready to create social policy around this "fact."
Our conversations went something similar to this:
"Well, that's a very intriguing idea," I said. "It's a hypothesis, and it's interesting to think how we would test the hypothesis. First, we would have to study big populations of women to see if, in fact, those born to older women fared better as you suspect. Then we would have to try to separate out the difference between nurture and nature, that is, the way an older women raises children versus the way a teen raises children could explain much of that difference. If nurture did not seem to explain the difference, then perhaps we could examine the eggs from sisters of greatly different ages and see in what ways those eggs had physically changed with time so we'd have a plausible biological explana--"
At this point, my acquaintance was not only bored but obviously annoyed. "It's not a science thing. You can't approach this as a scientist because you cannot uncover the workings going on here. I am talking about the life experiences of the woman having an effect on the essence of the egg itself, which produces an inherently more sophisticated embryo. … "
"What exactly is the essence? What is it made of? DNA? Mitochondria? Cytoplasm?"
"No! You are such a scientist! It can't be detected physically. It is the spiritual component of persons, its energy."
"Hmmmm. Energy is measurable. … "
"Not this kind of energy."
My acquaintances are very intelligent, have degrees from respectable universities, and have come up with this very interesting hypothesis. They were trying to discover and understand the workings of the world, but they made a series of errors that will prevent their idea from being accepted by a learned audience: they decided the idea was obviously true without submitting it first to valid testing (indeed, any testing at all). That is, they leapt straight from hypothesis to conviction, mistakenly thinking that a belief in understanding how the world works and valid science were separable. They did not concern themselves with the small numbers of women involved in their initial observations; had not considered other possible explanations for the effects they thought they saw; were unperturbed that, for their theory to work, they had to suspend or rewrite natural physical law; were instantly certain the "essence" and "energy" existed, even though they were not detectable; and held firm to their belief that this idea was "above" scrutiny and reason. They were so attached to the idea that they became annoyed that it would be called into question.
In order to discover a truth about how the world works, a hypothesis is a beginning, not an end. Statistically, an armchair hypothesis has a short life expectancy in the no-nonsense arena of science, and a hypothesis sent up primarily for its emotional appeal is likely to come to an end as inglorious as it is swift.
"We are certainly not to relinquish the evidence of experiments for the sake of dreams and vain fictions of our own devising."
--Sir Isaac Newton (1642-1727), professor of mathematics, Cambridge University,
Principia, 1687
When we observe phenomena in our daily life, we are free to speculate on cause and effect. But speculation has virtually no merit. So we read up on the issue, search for similar observations that are verifiable by rigorous criteria, and make a hypothesis -- an educated guess. The hypothesis is then put to the test using a specific method of quantifying, controlling confounding variables, randomization, blinding participants from their own biases… The results of the study are evaluated in light of all reasonable strengths and weaknesses, and all plausible explanations are offered for the observations. The study is then submitted to withering scrutiny by others. This process is repeated by unrelated parties to demonstrate reproducibility and to explain any variations in outcome. And the process produces only a tentative conclusion, subject to unending reconsideration and earns a level of confidence -- seldom certainty -- that is only proportionate to the strength of the evidence in hand.
Such is the process from neat idea to fact. It is a fundamental methodology too unfamiliar to many Americans to be bothered with.
It is not clear that the current generation of students in the U.S. knows less science than previous generations. We know only that dozens of countries have, in the last fifteen years, caught up with and surpassed us in scientific literacy and achievement. Nonetheless, it is a reminder that we are in no position to assume superiority - and perhaps not even competence fitting our potential -- in a realm of human endeavor that directly sustains and advances our democracy. Science is an amazingly fruitful endeavor in its own right. But the disciple of mind, the insistence on valid evidence, and the rigor of the experimental method are tools that are applicable to many issues outside of benchtop science, most especially to the creation and application of justice in an enlightened society. We should tremble at the signs that our society's grasp of science and scientific thought may be slipping and that so much of our society is indifferent or even pleased at the prospect.