IN 1962 the philosopher of science Thomas Kuhn published the book The Structure of Scientific Revolutions.

In it he claimed that science progressed through a series of revolutions in scientific knowledge for which he coined the term “paradigm shifts.” 

These paradigm shifts were radical new ideas and knowledge that subverted the previous understanding in a field of science and set it on an entirely new path.

A classic example of this would be the theory of Copernican heliocentrism, when the Sun replaced the Earth as the centre of the universe. 

The discovery of the double helix structure of DNA by Rosalind Franklin, Maurice Wilkins, James Watson and Francis Crick is another example.

The geometrical understanding of DNA produced an explosion of deeper understanding of genes that continues today.

This idea, that science is sporadically disrupted by innovative novel ideas, underlies a recent study that tries to measure the level of “disruptiveness” over time in 25 million published research papers across many different scientific fields.

The analysis required defining an easily calculable numerical metric of disruptiveness, since it wouldn’t be possible to read and understand so many different papers.

The metric that was invented for this purpose was calculated for each paper by looking at whether subsequent papers also cite the academic literature from before this paper was published, or stop citing previous work altogether.

The idea is that the paradigm shift caused by a disruptive paper makes previous work redundant, or at least less relevant to post-disruption research.

The recent analysis found that in most scientific fields the average “disruption metric” of papers had decreased over time.

Science is, on average, less disruptive than it used to be. Drilling down into their data, however, they found that the actual number of disruptive papers was roughly constant, but that disruptive papers as a proportion of all papers had hugely decreased.

Many more papers are being published that are not disruptive, diluting how disruptive papers are on average.

The study provoked a lot of responses from active scientists on social media platforms such as Twitter and Mastodon.

Some voiced concerns that this metric may not capture disruption very well.

Different types of papers are cited for different reasons: review papers aim to collect and evaluate knowledge in a certain field and can become the sole citation shorthand for a large body of accepted work, without being themselves disruptive (or novel).

Another theory about the reductions in disruptive science is that we have solved the “low-hanging fruit,” or easier problems, and are left with considerably harder topics to research such as the origin of life or the nature of the universe.

In some fields collecting data might require the co-operation of thousands of scientists over many years to build particle accelerators or telescopes.

During this time, multiple theories might exist that do not conclusively upend previously held beliefs.

Many productive and important ideas in complex fields genuinely seem to be developed through dialogue with, rather than defeat of, previous ones.

However, there’s good reason to doubt the idea that truly novel results are harder to get now.

Reduction in disruption has been remarkably similar in most fields over time. If this were due to solving easier low-hanging problems then you would expect the disruption in different (older versus newer) fields to decline at different times and rates.

Cultures and technologies of citation seem also likely to play a role in the evolution of the disruption metric.

Even if this numerical metric does not capture changes in disruption very well, it seems to have spoken to an uneasiness many scientists feel about the structure of science.

Many scientists responded by saying that the analysis corresponded with what they felt was a stifling of creativity and novelty that is inherent in the current way that science is funded.

Scientists compete for grants from research councils and funding bodies and part of this process involves writing a grant application and defending the feasibility of achieving your project goals.

The more radical and disruptive your project idea, the less clear how feasible it will be to achieve your goals and the harder it will be to win funding.

This feasibility requirement biases funded science towards achievable goals, and certainly to measurable outputs such as published papers, even when the research results were not groundbreaking.

Scientists, in response to the idea that research is less disrupted now than it ever has been, highlighted the grant-allocation system as likely to produce cautious research based on incremental gains , rather than disruptive revolutionary discoveries.

The mindset of funding bodies is part of a wider mindset of “value for money” in higher education that prioritises monetary return in the form of patents, partnerships with industry, and maximising the number of new students.

Science departments have been largely sheltered from the relentless cuts that have befallen humanities departments, whose research often does not have value that is easily expressed in monetary terms.

But this does not mean that science has remained untouched by the subsumption of education into a profitable enterprise.

What is lost, in this system in which new ideas can be systematically bought, is the immeasurability of radical imagination. 

In fact, another key factor in the change may be that so much more science is done these days, by so many more people, using so much more money.

If we believe that this research effort can be valuable and meaningful, then the failure to produce correspondingly more genuinely new ideas appears to be a failure to capture the essence of novel research itself.

This could be a real problem. If scientific thinking becomes stale and captured by the interests of capital, we lose some of our ability to dream of utopias.