Tag Archives: statistical probability

Statistical probability vs. baseline plausibility

John Edward Terrell


WE ARE LIVING IN INTERESTING TIMES. Pundits are saying many of us in the United States are no longer able to distinguish between fact and fiction. Globally speaking and equally alarming, the world may be returning to the Dark Ages before the Renaissance when faith rather than informed skepticism ruled the day.

Figure 1. Despite evidence to the contrary, some today deny that the earth’s climate has changed radically since the start of the Industrial Revolution. Source: https://commons.wikimedia.org/wiki/File:Evidence_CO2.jpg

Drill down below the level of popular journalism and social media chatter, and you will discover that even scientists today stand accused of undermining popular faith in facts & figures. Christie Aschwanden at FiveThirtyEight is one of many science writers reporting that scientists have apparently been using statistical tests of significance (in particular, p-values)[1] to decide how true their results may be and whether they should write up their findings for publication. Yet, as Aschwanden observes: “The p-value only tells you something about the probability of seeing your results given a particular hypothetical explanation—it cannot tell you the probability that the results are true or whether they’re due to random chance.”

Drill down yet deeper below the level of common everyday understanding and it turns out statisticians, too, are adding to our collective uncertainty about who and what to believe in. Apparently they no longer agree on how to do what they do, statistically speaking.[2] If they aren’t sure, how can mortals like ourselves decide who to believe and what’s what?

What’s to be done?

If it is true that sweet reason is on the wane in the world today, what can be done—borrowing from a recent presidential campaign—to make America think again?

A conventional response would be that we must buckle down and commit ourselves anew to paying proper attention to data—solid, down-to-earth facts & figures. Surely this isn’t such an impossible task? And to help us out, let’s not forget that modern statistical tests of significance—however contested nowadays they may be by professional statisticians—were developed by experts years ago to help us conclude whether we have enough facts on the table to decide what they are showing us. Therefore, shouldn’t combining due diligence for real facts with properly employed statistical testing be sufficient to get us past the current “post-factual” crisis?

Unfortunately such a seemingly self-evident way of confronting today’s reported decline in the quality of human wisdom and collective insight is not only too simple, but also naive. Here’s why.

The mind’s artful creativity

The biologist François Jacob—he shared Nobel Prize in Medicine in 1965 with Jacques Monod and André Lwoff—titled his 1982 book about evolution The Possible & the Actual.[3] This title seems misleading. Jacob is not advocating in the three beautifully written essays in this slim volume for a traditional no-nonsense view of science as the rock-solid quest for the actual—for truth and certainty—although in his preface he does say:

. . . human life always involves a continuous dialogue between the possible and the actual. A subtle mixture of belief, knowledge, and imagination builds before us an ever changing picture of the possible. It is on this image that we mold our desires and fears. It is to this possible that we adjust our behavior and actions. In a way, such human activities as politics, art, and science can be viewed as particular ways of conducting this dialogue between the possible and the actual, each one with its own rules.

Yet Jacob’s message for us is not as elementary as this statement may suggest. As a committed evolutionist, he has in mind a decidedly utilitarian view of this purported dialogue. In his own words:

In lower vertebrates, sensory information is converted into moto-nervous information in a rigid way. Such animals appear to live in a world of global stimuli closely linked to processes of appropriate responses, what ethnologists call “innate releasing mechanisms.” In contrast, in birds and even more so in mammals, the enormous amount of information coming from the environment is sorted out by sense organs and processed by the brain, which produces a simplified and useful representation of the external world. The brain functions, not by recording an exact image of the world taken as a metaphysical truth, but by creating its own picture.

From this perspective, therefore, asking how truthfully your brain is taking in the world around you is off the mark. The real issue is how usefully it is doing so. “The external reality, the ‘reality’ of which we all have intuitive knowledge, thus appears as a creation of the nervous system. It is, in a way, a possible world, a model allowing the organism to handle the bulk of incoming information and make it useful for its everyday life.”

The plausible and the possible 

Over the years since the publication of The Possible & the Actual, research in cognitive psychology and neuroscience has done much to clarify how—and how selectively—the human mind constructs what Jacob calls the particular “possible world” within which it dwells.[4] Seen from your nervous system’s point of view, the job we are doing when we are dealing with the world isn’t drawing the line, so to speak, between what Jacob called the possible and the actual, but rather between the plausible and the possible —between what our experiences have taught us to expect to find “out there” in the real world and what might possibly be there based instead on logic or fantasy rather than facts & figures (see fig. 2).

Figure 2. From your nervous system’s point of view, the task you are constantly facing when you are dealing with the world around you is where to draw the line between the possible and the plausible, not the possible and the actual.

If so, then as Daniel Kahneman and others have been saying for years, where we draw the line between the plausible and the possible depends not only on facts & figures bolstered perhaps by statistical analyses, but also on our previous experiences, biases, and prior assumptions about the world and how it works that may not only be all but impossible to put on the table but may not even be part of our conscious awareness.[5]

Conclusion

Headlines lamenting that many of us have seemingly lost the ability to distinguish fact from fiction may be catchy and provocative, but we know enough now about how the human mind constructs its sense of reality to conclude—along with Jacob, Kahneman and others—that paying proper attention to facts & figures is only part of the challenge we face nowadays. It can be argued that how science itself, for example, is conventionally done focuses too much on weighing hypotheses in the light of evidence, and not enough on where hypotheses, so to speak, come from in the first place (see fig. 3).

Figure 3. Science is not just an interplay between evidence and hypotheses, but also with the assumptions, beliefs, and prejudices that make some hypotheses seemly more worthy of study and experimentation than others.

The operative question we need to be asking these days, therefore, is not only the statistician’s old “p-value” question “Do I have enough evidence in favor of my hypothesis to reject the null hypothesis?” We must be attending also more than currently seems popular to the broader question “Why do I find my favored hypothesis plausible enough to spend time and money taking it seriously?”

References

[1] Nuzzo, Regina. “Statistical errors.” Nature 506, no. 7487 (2014): 150-152.

[2] Rodgers, Joseph Lee. “Moving in Parallel toward a Modern Modeling Epistemology: Bayes Factors and Frequentist Modeling Methods.” Multivariate Behavioral Research 51, no. 1 (2016): 30-34.

[3] Jacob, François. The Possible and the Actual. Pantheon, 1982.

[4] Pouget, Alexandre, Jeffrey M. Beck, Wei Ji Ma, and Peter E. Latham. “Probabilistic brains: knowns and unknowns.” Nature neuroscience 16, no. 9 (2013): 1170-1178.

[5] Kahneman, Daniel, and Shane Frederick. “A model of heuristic judgment.” The Cambridge handbook of thinking and reasoning (2005): 267-293.

Ascoli, Giorgio A., Matthew M. Botvinick, Richards J. Heuer, and Rajan Bhattacharyya. “Neurocognitive models of sense-making.” Biologically Inspired Cognitive Architectures 8 (2014): 82-89.


John Edward Terrell is Regenstein Curator of Pacific Anthropology at The Field Museum, Chicago, IL 60605. His latest book A Talent for Friendship: Rediscovery of a Remarkable Trait was published on December 1, 2014 by Oxford University Press. Email address: terrell[at]fieldmuseum.org

© 2017 John Edward Terrell. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author(s) and source are credited. The statements and opinions expressed are those of the author(s) and do not constitute official statements or positions of the Editors and others associated with SCIENCE DIALOGUES.