Mathematical Methods and Research as a Community

The famous ancient historian Walter Scheidel has reviewed a book on mathematical models of the economy of the Roman empire.

The current system of academic training, recruitment and promotion is not well equipped to recognize work that is routinely collaborative, may result in electronic outputs rather than traditional deliverables, and is not overtly focused on the monograph as the basic coin of the realm. All that makes it hard to reconcile with norms and expectations that are deeply entrenched in the academic humanities, most notably in the United States where institutionalized individualism and fetishization of the little-read book rule supreme. Academic incentive structures will need to be tweaked in favor of collaborative and non-traditional work to give simulation studies a chance to flourish.

Some of my gentle readers may not know that in ancient world studies we have a situation where to make a bibliography count for academic promotion, we have to print a few hundred copies and sell them to libraries where they collect dust while researchers check the website with PDFs or a searchable database. Rachel Mairs’ Hellenistic Far East Bibliography faces this barrier, so does the ETCSL. And peer-reviewed publications in ancient history and philology are still expected to be written by one or two authors, whereas in natural science there are often a dozen or more authors who contribute different specialized skills (perhaps one performs a chemical test, another writes the software, a third does most of the writing, and a fourth manages the project). But I see a big problem with pushing to focus on understanding the ancient world through mathematical models.

Right now all areas of research are in an epistemic crisis because our networks of verified trust have broken down. This is obvious with public health policy and social psychology but as far back as 2009, Jona Lendering noticed that 37 of 50 common errors about antiquity came from trained experts writing outside their area of expertise (eg. an expert in Greek literature writing about Roman archaeology). For many different reasons, these mathematical models are especially hard for outsiders to evaluate. They certainly require math and programming skills which not many students of the ancient world possess, but often they use proprietary software packages, large amounts of computing power, or the kind of big data which is bad data. And if you want to spiral closer and closer to the truth, you want to use methods which as many peers can evaluate as possible.

I see people trying to put history before the Iron Age into absolute time by using complicated statistical techniques on incomplete sequences of tree rings (dendrochronology) and ambiguous radiocarbon dates. This is very ambitious, but its very easy to assume your conclusion, and few people have the skills and time to see if the methods work. Already in 1991, it was very difficult to sort out the evidence for chronology in the area from Sweden to Egypt and conclusively demonstrate either Petrie’s Egyptian chronology or its challengers.

Discussions of research such as Gray and Atkinson on Proto-Indo-European are often frustrating because many critics did not have the language to express the mathematical problems with their argument and the journal did not understand the logical or philological problems. (Papers in natural science are also structured differently than papers in the historical sciences, so you have to learn to read each genre). The rise of twitter did not help things since it rewards snappy witticisms over moderate and reasoned criticism (has anyone seen a clear update on the theory of a cosmic airburst over Tall el-Hamman which avoids insinuations and overstatement and just focuses on the evidence and methods?)

Training more archaeologists and historians in math and statistics could be a good idea, but we are already struggling to spot holes in arguments built with our existing methods. It took 50 years before someone said in print that the glued linen theory did not match any surviving armour before 1970, or any description of armour being made before 1860, because most people interested in ancient Greek warfare are not archaeologists or armour scholars. Until we improve our systems of mutual criticism, adding more methods will just create more opportunities for a bad argument to slip through because nobody can see and explain the problems.

Scheidel enthuses:

The payoff promises to be considerable: novel pathways of analysis as well as consilience with social science fields in terms of research design and hypothesis testing.

But the quantitative social sciences produce more manure than a pig farm, even though statistical methods are their main tools. They do not produce results you can rely on without a lot of inspection. Expecting archaeologists who dabble in statistics to use them better than psychologists or sociologists is very optimistic, and so is expecting ancient world studies to clean up after bad studies when psychologists and sociologists and economists have failed. I think ancient world studies has things to offer quantitative social science, from case studies before the 19th century to our care with our data and curiosity about where those numbers come from and how the things we can measures relate to the things we want to study.

So I would agree that administrators should give more credit to online publications and post-publication peer review (if your colleagues all use a resource, that is very strong evidence that they think it is sound and useful, much stronger than if two colleagues and an editor agreed it was publishable). I also agree that academic training in the historical sciences encourages people to try to do everything themselves, and that this is inefficient and leads to overwork. Very few people are good researchers, and good popularizers, and good at keeping an audience on social media, in addition to all the basic academic tasks like teaching and department administration. Encouraging a more collaborative approach would be of value. I even agree that the archaeology of the Roman empire can produce something that is much easier to call data than ancient texts or ancient paintings and sculptures.

But if we want to learn true things about the past, I think we should put more resources into collaborative criticism and dialogue between different types of researchers, and not in to ‘big ideas’ that fewer and fewer people can evaluate. The ‘big ideas’ might help ancient world studies compete with boastful disciplines like economics, but without stronger criticism, they won’t lead us closer to how it really is, or convince other thoughtful skeptical people that our work and methods have value. As Jona Lendering warned us in 2009, lets work harder at suppressing the false even if finding new truths is more fun.

As I will say at the end of this year, to launch really big impressive projects, you have to trust a lot of people and some of them will cheat you. Dan Davies convinced me of this in his book on the economics of fraud. I am happy to focus on my less ambitious and more skeptical work, helping people understand what ancient Greek swords or Babylonian temples were like rather than developing a grand theory of the Roman economy. That will never make me famous, but it will never lure me into telling millions of people a plausible lie.

PS. For the record, I have a degree in computer science (so about two years each of calculus, linear algebra, graph theory, statistics, and so on)

Further Reading: Kostas Vlassopoulos shows how using numbers can be ‘truthy’ rather than scientific in another BMCR of a book I have not read.

Edit 2023-08-05: Stanford’s ORBIS model is a good example of something you can do using Roman archaeology which is really amazingly cool but where the results may not be as accurate as they seem (can’t comment directly not my specialty) https://orbis.stanford.edu/

(scheduled 10 July 2023 based on a Mastodon thread)