HISTORIANS must work with limited information about the past.

First, what is set down is a biased subset to begin with. The experiences of the working class in England before the Industrial Revolution, for example, are much harder to reconstruct than the royal court.

Second, even written records can be lost, crumble away in attics, or be burned in library fires. So the evidence that does survive today from the past is a proportion of what once existed. But what is that proportion?

Research published last month in the journal Science tried to estimate this proportion for medieval manuscripts.

The Gutenberg press dates to around 1440. Before that point, if a particular story from medieval literature was to be passed on through history, it did so as a handwritten manuscript.

This could happen multiple times. Each manuscript containing the story is known as a “witness.”

The team of researchers included experts in each of the medieval literatures studied, as well as the statistician Anne Chao.

Since the 1980s, Chao has worked on statistical methods for estimating the true number of species in an ecosystem.

In ecology, a common problem is that we want to know the total “richness” of a community: how many different species are present.

The ecosystem may have been surveyed a few times, but always in a limited way. How can you estimate what total number of species actually is from many varying observations?

Consider looking out the window into a garden. Imagine that the garden only contains sparrows. If you look out 100 times, you will only ever see sparrows. The total richness of the garden is very low.

Now imagine a different garden which contains many different bird species. If you do the same experiment, then each time you look out the window you might see a different species that you haven’t seen before. The more times you look, the more of the total species you will have seen.

However, this will be more true at the start of your observations, because you are bound to see more common species more often.

You will observe some species multiple times, but some rare species only once.

Can you ever be sure that you’ve seen all the species? In any real experiment, some species are likely to remain unseen. Since you can’t keep observing forever, the problem is how to deal with this undersampling.

Chao’s work is all about this problem. Her insight was that the proportion of singletons (species you only ever observed once) and doubletons (those you have observed twice) contains key statistical information that can be related to the total number of species with a simple formula. Chao’s formula is now widely used in ecology.

Remarkably, the formula gives a valid lowest possible value for any diverse and undersampled group of objects that can be grouped into types.

The types don’t have to be species. The same mathematics can be adapted to apply to the survival of medieval manuscripts.

For this work, the researchers tallied up the known manuscripts that contained different stories within the genre of heroic and chivalric fiction, which existed in many medieval languages. They chose six languages: French, German, Dutch, Irish, Icelandic and English.

For each work, they had a number of “observations” of it for each different manuscript in which it appeared.

Like rare birds, some rare stories only survived in a single manuscript, but more common stories survived in multiple manuscripts.

Previous estimates of survival rates of these works have used library catalogues. By looking at lists of works held in a library and counting up how many copies exist today, it has been estimated that for the holy Roman empire the overall survival rate of general purpose manuscripts was around 7 per cent.

For this work, the researchers applied Chao’s methods to their data. This allowed them to estimate the total original number of works and therefore their survival rate to the present day.

They found different survival rates depending on the language of the works, ranging from less than 5 per cent for English to nearly 20 per cent for Irish. The average figure is roughly consistent with the previous 7 per cent estimate.

Why the differences between languages? One of the suggestions is that the Norman conquest of England led to the neglect of English in the years that followed and a greater loss of associated works compared to other medieval languages.

Although other explanations are possible, this would be consistent with the destructive effect that invasion and colonisation can have on existing languages. History is written by the victors and usually in their language too.

For Irish, the researchers hypothesise that works may have had a higher survival rate because of the way medieval manuscripts were transferred there.

They noticed that when they compared the “evenness” of distribution of works across manuscripts, it is more common in both Irish and Icelandic to find the same stories present across many manuscripts. The stories were simply reproduced in greater quantities.

This greater redundancy may have played an important role in greater survival, because for a story to be forgotten forever required many manuscripts to be lost.

This research is a fascinating example of how methods can be reapplied across disciplines to change how we understand the past. It’s also an excellent reminder of how easy it is to forget  history.

Many stories are never recorded and the vast majority of those that are recorded will likely be lost as well. It’s a reminder of the importance of preserving the limited historical evidence we have on topics that have often been deliberately forgotten.

When a tiny fraction of all the stories and histories will survive, it’s up to us to fight for the preservation of those tales and records that will inspire future generations to create a better world.

You can read the original paper in full at www.mstar.link/3tK41hc.