Chapter XXV: Baselines And The Null Hypothesis | The Philosophy Of Science by Steven Gussman [1st Edition]

        “There are no solutions, there are only trade-offs.”

        – Thomas Sowell^I

        “What can be asserted without evidence can also be dismissed without evidence.”
        – Christopher Hitchens^II

        “Extraordinary claims require extraordinary evidence.”
        – Pierre Simon Laplace (as paraphrased by Carl Sagan)^III

        Baselines (or base-rates) are an acknowledgment of the fact that, totally aside from a given intervention, many effects are occurring in some normal rate all of the time.  Imagine, for example, that you give drug x to 10,000 people for a period of a year in order to figure out if it is dangerous.  Then imagine that 10 people in the study died.  One might be tempted to worry that the drug has a 10 / 10,000 = 0.1% chance of killing those who take it!^IV  But people die for a variety of reasons, including from natural causes associated with aging; one would need to control for this to see if more (or less!) than the expected base-rate of deaths occurred in the group.  For starters, one can use external death statistics and see if their study-rate deviates from the general statistic.  An even better, but still elegant, way to control for this (and many other baseline effects) is to have an entire control group in the study who is not given drug x, so that you have measured your own a baseline to compare your study group's values against (which might potentially control for other confounds, or alternative co-variables, such as the characteristics of people who might be more likely to join this particular study).  More generally, too often (especially in politics), people make the mistake of comparing the outcomes supposedly attributable to some effect, to some utopian vision in their head: an unrealistic baseline in which nobody dies, no one is impoverished, and no one misbehaves.  Doing so always leads to inaccurate and biased conclusions.  One has to be realistic about how the world already is, normally, before they begin attributing causes to effects, let alone thinking about what it means to change it in the real world.

        Similarly, the null hypothesis is the hypothesis we should take to be provisionally true by default, in lieu of any evidence.  Very often, perhaps most often, there is no null hypothesis: in lieu of evidence in favor of a particular hypothesis, we should remain in equipoise (or remain agnostic towards the explanation).  In other cases (either due to the logic at play, hints from prior knowledge, or otherwise because we need to act as if one or another hypothesis is true), however, one situation should be seen as more likely a-priori, with the greater burden of evidence on establishing the alternative (for example, when it comes to medical treatments especially with pharmaceuticals, we always follow the principle of precaution by taking the null hypothesis to be that a new proposal is relatively ineffective and unsafe until proven otherwise).^V  As a trivial example, when you already have a working theory (which has passed empirical tests), but someone puts forth a new, competing hypothesis (perhaps one which explains what the current theory explains, and more—or perhaps one which claims the other theory was barking up the wrong tree and only incidentally passed empirical tests thus far), the likelihood is that the old theory is true, and the new theory false, ahead of time (for the very simple reason that most new ideas are untrue^VI).  The mere presence of a new competing hypothesis does not, in itself, call into question the previous theory (which has already passed empirical tests).  Other times, common sense will dictate that the null hypothesis is at least something other than the proposed explanation, otherwise known as failing to pass “the sniff test” (which can easily be abused if one is not careful). “Extraordinary claims require extraordinary evidence,” as Sagan would say,^VII and if someone proposes, for example, that homo sapiens' affinity for sugary foods was caused by the prevalence of a pre-historic virus which thrived in sugar-rich bodies, and so caused its hosts to eat sugar, you are free and wise to dismiss this out of hand in favor of a simpler explanation, say, that sugar was useful to our ancestors in their pursuit of energy (always keeping an open mind that empirical evidence down the road might bring the less obvious hypothesis into more serious play).  In fact, one may provisionally dismiss certain hypotheses even in lieu of a preferred alternative; as journalist Christopher Hitchens was known for saying, “what can be asserted without evidence can also be dismissed without evidence.”^VIII  Philosopher Bertrand Russel used the example of the celestial teapot: even though you cannot be sure there is no celestial teapot orbiting the sun, you would be wise to doubt it out of hand; the burden of proof is on those making the novel claim (especially when it is extraordinary).^IX  It was posited without evidence, and the null hypothesis is clearly that nothing but ice, gas, and rocks orbit stars—that is, until evidence suggests differently.^X  Otherwise, all of scientists' time would be spent falsifying all of the random possible claims that could be made, rather than judiciously pursuing those fruitful lines of reasoning likely to progress the state of knowledge.^XI  More generally, the burden of evidence must be on the the claimant to establish their hypothesis, not on everyone else to falsify it.^XII  There is even a time-dynamic version of this known as regression to the mean, which basically means that even when the unlikely scenario does occur, it is most likely to be followed by a likely scenario.^XIII  That is, when the baseline is deviated from, it is most likely over time that it is nevertheless returned to (otherwise, it wouldn't be the baseline).  A scientist should recognize the difference between speculation and a theoretically (or empirically) motivated prediction, respecting that speculation may graduate as the process plays out.

Footnotes:

0. The Philosophy Of Science table of contents can be found, here (footnotephysicist.blogspot.com/2022/04/table-of-contents-philosophy-of-science.html).

I. See the “Empiricism” chapter which further cites “Thomas Sowell: There Are No Solutions, Only Trade-Offs” by Sowell (https://www.youtube.com/watch?v=3_EtIWmja-4) (0:25 – 0:37) and A Conflict Of Visions by Sowell (at least pp. 14, 17, 19).

II. See “Hitchens's Razor” (Wikipedia) (Accessed 11/27/2022) (https://en.wikipedia.org/wiki/Hitchens%27s_razor) which further cites God Is Not Great: How Religion Poisons Everything by Christopher Hitchens (Twelve Books) (2009) (pp. 258) (though I have not read the entire Wikipedia article nor this book). Shermer calls this Hitchens' Dictum, see for example his April 12^th, 2015 tweet: https://twitter.com/michaelshermer/status/587157023617196033?s=20&t=AK8FgUiFpvPnljXogtB2fQ.

III. See the "Mechanical Philosophy" chapter which further cites Cosmos by Sagan, Druyan, and Soter, “Sagan Standard” (https://en.wikipedia.org/wiki/Sagan_standard), and “Pierre Simon Laplace” (https://todayinsci.com/L/Laplace_PierreSimon/LaPlacePierreSimon-Quotations.htm).

IV. The chance of dying as a result of something once you have contracted it is known as the infection fatality rate, or IFR (or more generally, the case fatality rate, or CFR). This measure controls out the prevalence of the cause (there is somewhere between a 0% and 100% chance of encountering a given cause, and the overall chance of catching-and-dying is governed by the product of this prevalence P and case fatality rate CFR: P_Death = P × IFR).

V. See the “Conservativism And Progress” chapter which further cites “Precautionary Principle” (https://en.wikipedia.org/wiki/Precautionary_principle) which in turn cites “The Precautionary Principle Under Fire” by Read and O'Riordan (https://ueaeprints.uea.ac.uk/id/eprint/65524/1/Accepted_manuscript.pdf) (though I have not read the Wikipedia article beyond the opening definition, nor any of this paper)

VI. See The Demon Haunted World by Sagan (pp. 305).

VII. See this quotation at the top of the chapter, and footnote III for its citation.

VIII. See the quote at the top of this chapter, and footnote II for its citation..

IX. See The God Delusion by Dawkins (pp. 74-77, 94-95, 437) which further cites “Is There A God?” by Bertrand Russell (1952 / 1997) (though I have not read this work) and “Russell's Teapot” (Wikipedia) (accessed 11/28/2022) (https://en.wikipedia.org/wiki/Russell%27s_teapot) (though I have not read this entry).

X. See The God Delusion by Dawkins (pp. 74-77, 94-95, 437) which further cites “Is There A God?” by Russell.

XI. See The God Delusion by Dawkins (pp. 76).

XII. See The God Delusion by Dawkins (pp. 76). Of course it is healthy that anyone interested try to falsify a given idea (which may lead to its statistical verification), but this is not owed to the claimant; the burden of evidence cannot be on others, and the null hypothesis cannot favor a given man's pet hypotheses for no good reason.

XIII. See “Regression To The Mean” by James J. O'Donnell (Edge / Harper Perennial) (2017 / 2018) (https://www.edge.org/response-detail/27013) in This Idea Is Brilliant edited by Brockman (pp. 272) (though this essay begins with an absurd insinuation that science is subjective—see again the “Objectivity” chapter).