Chapter XXIV: peer review | The Philosophy Of Science by Steven Gussman [1st Edition]

        “The universe revealed by the relentless error-correcting mechanism of science was to him

        infinitely preferable to the untested assumptions of traditional belief.  For Carl, the 'spiritual' had to

        be rooted in natural reality.  He cherished those ideas about the cosmos that remained after the

        most rigorous experiment and observation.  Scientific insight made him feel something, a soaring

        sensation, a recognition that he could only compare to falling in love.”

        – Ann Druyan^I

        “Real peer review is what happens after you've passed the bull-shit thing called, 'peer review.'”
        – Eric Weinstein^II

        “Science is open mic night.”
        – Eric Weinstein^III

        As discussed previously, each and every scientist should strive to embody the scientific ethic and follow the scientific method.  However, some will be better than others, and no one will be perfect.  This is where the society of scientists comes in.  Cultures are the product of many people and admixtures of thought.  It is important that there is far more than one scientist at work on the scientific enterprise.  For one, scarcity of resources such as time are such that one man could not possibly be tasked with figuring it all out in one lifetime.  But even a single given result benefits greatly from the engagement of many researchers.  This collective acts as a control on many variables: intelligence, bias, and error being among them.^IV  The scientific method is self-correcting in large part because adversaries act as a check on each other's research: there is someone else trying to falsify every idea that another is trying to establish.  This rich marketplace of ideas, as long as it places a premium on veracity over other considerations, is a good environment for truth-seeking and sense-making.  The scientific norm is that one must publish their results (in the best case, making their raw data available as well) and arguments, leaving others free to conduct their own research or otherwise critique the original work.  And so it is this general sense of free inquiry and discussion that makes up small-p-r peer review.

        Today, when most people say, “Peer Review,” they really mean a narrow version of it: capital-P-R Peer Review.  This is the set of norms that, since the mid-twentieth century, have dominated professional academic research journals: for better or worse, much of day-to-day scientific research is conducted by PhDs associated with universities or think-tanks.^V  This research is Peer Reviewed (often anonymously) by one's colleagues in their field, before ultimately being rejected, or published in a scientific journal.^VI  It is only then that the real peer review begins.^VII  Peer Review is merely the normative process among scientific journals for editing and deciding whether to publish a given submitted paper.  This most-common version of the term is something of an abuse; it should probably be called something else, perhaps simply, “blind editing,” since it is in many ways closer to the job of a writer's editor at a publishing house (of course, it makes no sense to assume that a couple of fellow researchers could fully vet or falsify a new paper in the limited time before publication, which would suggest a false easiness to the scientific process).^VIII  There is nothing wrong in principle with institutions such as scientific journals requiring some kind of quality-control process (the content of their journals may well benefit from it, as intended).  But A. this process does not constitute real peer review—a long-term (technically unending, given the provisional nature of knowledge) process in which cooperating and competing thinkers establish the certainty of a given hypothesis by using the epistemological tools outlined in this book; and B. it is in some cases in-practice parasitized by ulterior and misguided motives.^IX  Often times, papers will be held up in Peer Review for years, delaying scientific progress.^X  Some un-scientific concerns may be arbitrary standards such as formatting, or that one's reviewer simply prefers a different research direction or result (maybe more in line with their own publications!).  Such Peer Review may ultimately acts less like quality control (a vetting that the authors followed the scientific method with due diligence), and more as an elitist gate-keeper ensuing the adherence to the academic trends of a given field.^XI

        It is important that we attempt to falsify our own ideas and that others do as well: the more (and more diverse) flak a hypothesis or theory survives, the more certain of it we can be.  Especially as we pass knowledge down through the generations, new thinkers must continue to test and expand upon (or discard) old ideas in novel ways.  Only through the varied approaches (both theoretical and experimental) of many thinkers (especially the minority of geniuses) will the scientific project stay on track.  Equipped with the tools of the scientific method, just your reading and responding to this book (and any work you encounter) is part of the larger peer review process: read, think, experiment, and write!

Footnotes:

0. The Philosophy Of Science table of contents can be found, here (footnotephysicist.blogspot.com/2022/04/table-of-contents-philosophy-of-science.html).

I. See “Foreword” by Druyan in Cosmos by Sagan (pp. xviii).

II. See “The Problem With Peer Review - Eric Weinstein | The Portal Podcast Clips” uploaded by the official The Portal Clips YouTube account (The Portal) (2020) (https://www.youtube.com/watch?v=U5sRYsMjiAQ) (3:48 – 5:58) which is an excerpt from “Bret Weinstein On "The Portal" (w/ Host Eric Weinstein), Ep. #019 - The Prediction And The DISC.” by E. Weinstein and B. Weinstein (https://www.youtube.com/watch?v=JLb5hZLw44s) (1:32:19 – 1:34:18)

III. See "Science is Open Mic Night - With Eric Weinstein - Bret and Heather 22nd DarkHorse Podcast Livestream" by Bret Weinstein, Eric Weinstein, and Heather Heying (DarkHorse) (2020) (https://www.youtube.com/watch?v=e6_8W4E0W1w) (27:10 – 30:46).

IV. For a similar mechanism at play in meta-analyses, see “#84: Hey YouTube: Divide By Zero (Bret Weinstein & Heather Heying DarkHorse Livestream)” by B. Weinstein and H. Heying (https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5idXp6c3Byb3V0LmNvbS80MjQwNzUucnNz/episode/QnV6enNwcm91dC04NzMwNTYw?sa=X&ved=0CAUQkfYCahcKEwiY2uSX-Pn6AhUAAAAAHQAAAAAQcg) (40:38 – 1:23:45).

V. See E. Weinstein's (@EricRWeinstein) January 19^th, 2022 tweet: https://twitter.com/EricRWeinstein/status/1483860161965932544?s=20&t=ic4tEsgugXTN1IRzM-WfhA which further cites a fascinating Google Books Ngram graph of the use of the term “peer review” over the years: https://books.google.com/ngrams/graph?content=peer+review&year_start=1800&year_end=2019&corpus=26&smoothing=3 (which suggests the term hardly existed until an inflection point at around 1965. Despite the method's introduction in medical articles in 1731, JAMA didn't begin utilizing it until 1940 and The Lancet took until 1976 (the average of which 1958, in decent agreement with the 1965 value), see “The Birth Of Modern Peer Review” by Hadas Shema (Scientific American) (2014) (https://blogs.scientificamerican.com/information-culture/the-birth-of-modern-peer-review/) (an article linked to in E. Weinstein's January 26^th, 2020 tweet: https://twitter.com/EricRWeinstein/status/1221475903919648768?s=20&t=eDNPu35VDFuoXMRoFSmXaA). As E. Weinstein has pointed out, roughly all of the greatest scientific discoveries pre-date this: Johannes Kepler's laws of planetary motion (~1614), Sir Isaac Newton's discovery of calculus as well as the laws of physical motion, including gravitation (1687), Edward Jenner's first-ever vaccine (against smallpox, via cowpox; 1796), Charles Darwin's and Alfred Russel Wallace's theory of evolution by natural and sexual selection (1859), Albert Einstein's special theory of relativity (1905), Albert Einstein's general theory of relativity (an updated theory of gravity; 1915), the highly collaborative (and competitive) establishment of quantum physics (between 1900 and 1930), and even James Watson's, Francis Crick's, and Rosalind Franklin's discovery of the double-helix structure of DNA in 1953, which was explicitly not Peer-Reviewed despite my understanding of the journal Nature having had implemented the practice by that time (just to name a few), see E. Weinstein's July 7^th, 2021 tweet: https://twitter.com/EricRWeinstein/status/1412861546728853504?s=20&t=1e15-tU15tHHGePTD5c1Xw, “Kepler's Laws Of Planetary Motion” (Wikipedia) (accessed 11/26/2022) (https://en.wikipedia.org/wiki/Kepler%27s_laws_of_planetary_motion), “Philosophiæ Naturalis Principia Mathematica” (Wikipedia) (accessed 11/26/2022) (https://en.wikipedia.org/wiki/Philosophi%C3%A6_Naturalis_Principia_Mathematica), “Smallpox Vaccine” (Wikipedia) (accessed 11/26/2022) (https://en.wikipedia.org/wiki/Smallpox_vaccine), “Darwinism” (Wikipedia) (accessed 11/26/2022) (https://en.wikipedia.org/wiki/Darwinism), “Special Relativity” (Wikipedia) (accessed 11/26/2022) (https://en.wikipedia.org/wiki/Special_relativity), “General Relativity” (Wikipedia) (accessed 11/26/2022) (https://en.wikipedia.org/wiki/General_relativity), “Quantum Mechanics” (Wikipedia) (accessed 11/26/2022) (https://en.wikipedia.org/wiki/Quantum_mechanics), “Molecular Structure Of Nucleic Acids: A Structure For Deoxyribose Nucleic Acid” by James D. Watson and Francis H. C. Crick (Nature) (1953) (https://www.nature.com/articles/171737a0), and “The Problem With Peer Review - Eric Weinstein | The Portal Podcast Clips” by E. Weinstein and B. Weinstein (https://www.youtube.com/watch?v=U5sRYsMjiAQ) (5:12-5:27) (though I have not read these Wikipedia articles in full).

VI. See “The Birth Of Modern Peer Review” by Shema (https://blogs.scientificamerican.com/information-culture/the-birth-of-modern-peer-review/); "Bret And Heather 10th DarkHorse Podcast Livestream: SARS-CoV2--Unintelligent Design?" by B. Weinstein and H. Heying (https://www.youtube.com/watch?v=FKtsx0fZzzQ) (6:40 – 8:24, 53:52 – 57:48); “Bret Weinstein On "The Portal" (w/ Host Eric Weinstein), Ep. #019 - The Prediction And The DISC.” by E. Weinstein and B. Weinstein (https://www.youtube.com/watch?v=JLb5hZLw44s); and "E15 - The Evolutionary Lens With Bret Weinstein & Heather Heying | Explorer Modes & The Lab Hypothesis | DarkHorse Podcast" by Bret Weinstein and Heather Heying (Darkhorse) (2020) (https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5idXp6c3Byb3V0LmNvbS80MjQwNzUucnNz/episode/QnV6enNwcm91dC0zNzg5MDM1?sa=X&ved=0CAUQkfYCahcKEwiIieTpv837AhUAAAAAHQAAAAAQZg) (53:20 – 56:10).

VII. See “The Problem With Peer Review - Eric Weinstein | The Portal Podcast Clips” by E. Weinstein and B. Weinstein (https://www.youtube.com/watch?v=U5sRYsMjiAQ) (3:48 – 5:58); "Bret And Heather 5th DarkHorse Podcast Livestream: Why Covid-19 Kills, And How To Stay Alive." by Bret Weinstein and Heather Heying (DarkHorse) (2020) (https://www.youtube.com/watch?v=uPTiP714LZM) (19:18 – 21:33); “"Bret And Heather 10th DarkHorse Podcast Livestream: SARS-CoV2--Unintelligent Design?" by B. Weinstein with H. Heying (https://www.youtube.com/watch?v=FKtsx0fZzzQ) (53:52 – 57:48); “E15 - The Evolutionary Lens With Bret Weinstein & Heather Heying | Explorer Modes & The Lab Hypothesis | DarkHorse Podcast” by B. Weinstein and H. Heying (https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5idXp6c3Byb3V0LmNvbS80MjQwNzUucnNz/episode/QnV6enNwcm91dC0zNzg5MDM1?sa=X&ved=0CAUQkfYCahcKEwiIieTpv837AhUAAAAAHQAAAAAQZg) (53:20 – 56:10); and “Bret And Heather 16th DarkHorse Podcast Livestream: Meaning, Notions, & Scientific Commotions” by B. Weinstein and H. Heying (https://www.youtube.com/watch?v=QvljruLDhxY) (0:59 – 51:37, 1:03:00 – 1:06:07).

VIII. See “Bret And Heather 16th DarkHorse Podcast Livestream: Meaning, Notions, & Scientific Commotions” by B. Weinstein and H. Heying (https://www.youtube.com/watch?v=QvljruLDhxY) (0:59 – 51:37).

IX. See “Bret Weinstein On "The Portal" (w/ Host Eric Weinstein), Ep. #019 - The Prediction And The DISC.” by E. Weinstein and B. Weinstein (https://www.youtube.com/watch?v=JLb5hZLw44s); “Bret And Heather 6th Live Stream: Death And Peer Review - DarkHorse Podcast” by B. Weinstein and H. Heying (https://www.youtube.com/watch?v=zc6nOphi0yE) (30:40 – 59:56, 1:35:30 – 1:39:35); "Bret And Heather 10th DarkHorse Podcast Livestream: SARS-CoV2--Unintelligent Design?" by B. Weinstein with H. Heying (https://www.youtube.com/watch?v=FKtsx0fZzzQ) (6:40 – 8:24, 34:33 – 37:27, 53:52 – 57:48); “Bret And Heather 16th DarkHorse Podcast Livestream: Meaning, Notions, & Scientific Commotions” by B. Weinstein and H. Heying (https://www.youtube.com/watch?v=QvljruLDhxY) (0:59 – 51:37, 1:03:00 – 1:06:07); “Bret And Heather 87th DarkHorse Podcast Livestream: We Must Drive This Virus To Extinction” by B. Weinstein and H. Heying (https://www.youtube.com/watch?v=QsUvr8s0qEk&t) (28:50 – 40:07); and Viral by Chan and Ridley.

X. See “Bret And Heather 5th DarkHorse Podcast Livestream: Why Covid-19 Kills, And How To Stay Alive.” by B. Weinstein and H. Heying (https://www.youtube.com/watch?v=uPTiP714LZM) (19:18 – 21:33) and “E15 - The Evolutionary Lens With Bret Weinstein & Heather Heying | Explorer Modes & The Lab Hypothesis | DarkHorse Podcast” by B. Weinstein and H. Heying (https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5idXp6c3Byb3V0LmNvbS80MjQwNzUucnNz/episode/QnV6enNwcm91dC0zNzg5MDM1?sa=X&ved=0CAUQkfYCahcKEwiIieTpv837AhUAAAAAHQAAAAAQZg) (53:20 – 56:10).

XI. See “Bret Weinstein On "The Portal" (w/ Host Eric Weinstein), Ep. #019 - The Prediction And The DISC.” by E. Weinstein and B. Weinstein (https://www.youtube.com/watch?v=JLb5hZLw44s); “Bret And Heather 6th Live Stream: Death And Peer Review - DarkHorse Podcast” by B. Weinstein and H. Heying (https://www.youtube.com/watch?v=zc6nOphi0yE) (30:40 – 59:56, 1:35:30 – 1:39:35); "Bret And Heather 10th DarkHorse Podcast Livestream: SARS-CoV2--Unintelligent Design?" by B. Weinstein with H. Heying (https://www.youtube.com/watch?v=FKtsx0fZzzQ) (6:40 – 8:24, 34:33 – 37:27, 53:52 – 57:48); “Bret And Heather 16th DarkHorse Podcast Livestream: Meaning, Notions, & Scientific Commotions” by B. Weinstein and H. Heying (https://www.youtube.com/watch?v=QvljruLDhxY) (0:59 – 51:37, 1:03:00 – 1:06:07); “Bret And Heather 87th DarkHorse Podcast Livestream: We Must Drive This Virus To Extinction” by B. Weinstein and H. Heying (https://www.youtube.com/watch?v=QsUvr8s0qEk&t) (28:50 – 40:07); Viral by Chan and Ridley; my November 22^nd, 2020 tweet criticizing Nature (@NatureComms): https://twitter.com/schwinn3/status/1330379138327318528?s=20&t=1e15-tU15tHHGePTD5c1Xw; my first June 23^rd, 2021 re-tweet of E. Weinstein: https://twitter.com/schwinn3/status/1407643296500006916?s=20&t=1e15-tU15tHHGePTD5c1Xw; my second: https://twitter.com/schwinn3/status/1407644172971085825?s=20&t=1e15-tU15tHHGePTD5c1Xw; my July 8^th, 2021 retweet of E. Weinstein: https://twitter.com/schwinn3/status/1413186942624227331?s=20&t=1e15-tU15tHHGePTD5c1Xw; my July 14^th, 2021 retweet of evolutionary biologist Nichola Raihani (@nicholaraihani) in which we critique Peer-Reviewers perpetuating publication bias: https://twitter.com/schwinn3/status/1415366715978960906?s=20&t=1e15-tU15tHHGePTD5c1Xw; my July 9^th, 2022 tweet: https://twitter.com/schwinn3/status/1545696354474889216?s=20&t=1e15-tU15tHHGePTD5c1Xw; my August 21^st, 2022 tweet (and surrounding Twitter thread): https://twitter.com/schwinn3/status/1561387437976346629?s=20&t=1e15-tU15tHHGePTD5c1Xw; my October 5^th, 2022 retweet of psychologist Kaitlyn M. Werner (@kaitlynmwerner) (and surrounding Twitter thread) in which we criticized Peer-Reviewers for arguing that a result must be “novel” (which ignores the importance of replication): https://twitter.com/schwinn3/status/1577642299697381378?s=20&t=1e15-tU15tHHGePTD5c1Xw; and my October 25^th, 2022 Twitter thread criticizing JAMA for allowing an obviously absurd claim through Peer Review, in which “STD” was defined as a disease which only transmits through sex: https://twitter.com/schwinn3/status/1585009360702705664?s=20&t=1e15-tU15tHHGePTD5c1Xw.