Science as Verified Trust
The higher you rise in any hierarchy, the harder it is to get accurate feedback about your decisions because people are afraid to tell you the truth. I’ve worked with several (US) presidents. All have made big blunders. I’ve also known and written about CEOs of big corporations who have made terrible mistakes. In every case, they had flawed systems for getting useful, accurate and reliable feedback.
Robert Reich (some kind of former political appointee from the USA) https://www.theguardian.com/commentisfree/2022/apr/01/vladimir-putin-ukraine-truth-deniers-bad-decisions
There seems to be a lot of confusion about the role of trust in science or scholarship. Engineers such as Bill Nye and political propagandists throw around the phrase “trust the science”! On the other hand, the rationalists whom I mentioned last year brandish the Royal Society’s motto nullius in verba “Take nobody’s word for it” like a sword. I think both sides are working from some misconceptions about how science or scholarship work.
Lets say I want to back a claim with a passage in Herodotus. I probably start by reading a translation. If the details are important, I can check a Greek text. How do I know that someone did not make up the Greek? The Greek text has a critical apparatus showing any significant differences between the printed text and the manuscripts. That text was printed by someone whom I never met and may have died before I was born. The scholar who created it worked with the manuscripts, and when it was done it was reviewed by other people who had seen the manuscripts and could compare other printed editions. I can’t read abbreviated Byzantine handwriting, and I have not tracked down all of these manuscripts or read them closely to know which seem to be closest. I can’t look at one of those manuscripts and say when it was written. But I can bring science forward by leaning on the networks of trust and suspicion which produced those critical editions. And the people who created those editions could not move science forward in the areas I write about.
I don’t need to trust some sacred text or wise teacher to know what Herodotus said. I could in principle trace any claim about what he said to the manuscripts and to a detailed study of the Greek language which left no commonplace unchallenged. But in practice, I (and most other people interested in the ancient world) very rarely do so. What makes this scientific or scholarly is not that you do every step yourself. It is that every step of the argument has been checked by multiple independent people, so in most cases you can quickly see if those people disagree and then trust those preliminary steps. Science or scholarship is not about heroes who know every skill, its about systems of questioning and verification which let us provisionally assume that some things are true while we focus on something where we are not sure of the answer. As humanity’s body of knowledge increases, we have to rely more and more on these systems of questioning and verification. This is more powerful than those super-scholars, because different types of people with different strengths can focus on different parts of the problem. I would be bored stiff comparing manuscripts and the development of the Greek language and Greek handwriting to guess how what Herodotus wrote might be different from what any of the manuscripts say. But some people love that! I am not a metallurgist, but I can learn from metallurgists who have examined ancient iron (just like they rely on archaeologists and conservators to excavate that iron, to clean it, and to assign it to a period and a culture).
The rationalists have an idea that a virtuous thinker should be prepared to engage with and independently evaluate any claim, however strange and however far from their expertise. This sounds rather adolescent to me, like the spotty teenager who demanded that a MMA star fight him. Its not “an impressive signal of epistemic virtue” to approach every claim someone makes with an open mind. That is a way to get tricked into believing nonsense and supporting horrible acts (or at least to be tricked into spending vast amounts of time and work carefully evaluating claims which were created by quick and lazy methods). The prudent person carefully balances asking “who should I trust?” on most topics, and asking “what does the evidence best support?” on a few carefully chosen topics. As long as some people ask about evidence and pass on their results, society as a whole can gain new knowledge. Its like how we can shop without worrying that some of the peppercorns are random seeds painted black, or go to a bar trusting that the glasses are the size they say they are, the drinks have not been watered down, and that the active ingredient is alcohol not methanol. Our society has people and institutions which verify that grocers and publicans are selling what they claim to sell, and these people and institutions are so effective that the rest of us we can forget they exist.
In my next post on this topic, I will talk about deciding who to trust and when this system can break down.
Further Reading
- Karl Bode, “Disinfo Wars: Fixing the Media’s Fake News Problem,” The Reboot, 5 April 2021 https://thereboot.com/disinfo-wars-fixing-the-medias-fake-news-problem/
- Andre Costopoulos, “William Shatner Pseudo-Archaeology Breakdown,” ArcheoThoughts, 4 April 2022 https://archeothoughts.wordpress.com/2022/04/04/william-shatner-pseudo-archaeology-tweet-breakdown-straight-from-the-pseudoscicomm-handbook/
- John Fleck, “Cadiz, scientization, and how Dan Sarewitz almost drove me out of journalism.” (23 October 2015) http://www.inkstain.net/fleck/2015/10/cadiz-scientization-and-how-dan-sarewitz-almost-drove-me-out-of-journalism/
- Derek Lowe, “Expertology,” In the Pipeline, 30 June 2016 https://www.science.org/content/blog-post/expertology
- George Orwell, “How do You Know the Earth is Round?” (1946) http://alexpeak.com/twr/hdykteir/
- Robert N. Proctor and Londa Schiebinger (eds.), Agnotology: The Making and Unmaking of Ignorance, https://www.goodreads.com/book/show/2748888-agnotology
- Martin Rundkvist, “Skepticism and Informed Consensus,” Aardvarchaeology 24 December 2007 https://aardvarchaeology.wordpress.com/2007/12/24/skepticism-and-informed-consen/
- Ken Thompson, “Reflections on Trusting Trust.” Turing Award Lecture. Communications of the ACM, Volume 27, Number 8 (August 1984) pp. 761-763 https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf
- Matthew Ygeslias, https://www.slowboring.com/p/misinformation-myth
(written early 2022, scheduled 21 March 2022, updated 12 April 2022 – I might have written it differently after 24 February 2022)
Edit 2022-04-26: added link to Costopoulos
Edit 2024-02-12: see also Daniel C. Dennett, “What if I’m Wrong?” behavioralscientist.org, 12 October 2023 https://behavioralscientist.org/ive-been-thinking-daniel-dennett-what-if-im-wrong/
Edit 2024-02-25: trackback from https://statmodeling.stat.columbia.edu/2024/02/24/science-as-verified-trust/
I fear your description bears more resemblance to the sciences of my youth than to The Science of the present day.
Perhaps scholarship in the humanities is less corrupted. Or perhaps I’m a starry-eyed optimist.
I think it depends on the field. Psychology has always had deep problems (remember the old Feynmann essay about psychologists who taught their grad students not to try the base case before they did their intervention?), observational biology seems solid. One of the future posts in this series will talk about how this system can break down and lead to science in name only.
[…] now all areas of research are in an epistemic crisis because our networks of verified trust have broken down. This is obvious with public health policy and social psychology but as far back […]
[…] …”) But at some point, the market or the communication starts to fall apart without systems of mutual verification. As big web service companies get more desperate for ever-higher proffits, I think many of them […]
[…] in foreign language and hope nobody would prove that they did not exist. Science relies on verified trust but it is mainly enforced by educating students into a culture of research, the peer-review […]
[…] histories (these genres also tend to have a lot of blatant factual errors because they do not verify their trust as carefully as academic history) […]