‘By avoiding the loss of resources and increasing the impact of research, metascience benefits society as a whole’

The assessment was made by British researcher James Wilsdon, executive director of the Research on Research Institute, in an interview with Agência FAPESP.

By Karina Toledo  |  Agência FAPESP – James Wilsdon, a social scientist by training, has become one of the best-known names in “metascience,” a field that uses scientific methods to analyze the research system itself to increase its efficiency. He was in Brazil to participate in the seminar “Research on Research and Innovation Project: Indicators, Metrics, and Evidence of Impacts,” held at FAPESP and the State University of Campinas (UNICAMP) on July 10 and 11.

Wilsdon has held academic positions at British universities such as Sheffield, Sussex, and Lancaster. He has also worked in non-governmental organizations and think tanks and served as director of science policy at the Royal Society, the UK’s national academy of sciences.

He is currently a professor of research policy at University College London (UCL) and the executive director of UCL’s Research on Research Institute (RoRI), which he helped found in 2019 with the mission of “unlocking the potential of the USD 3 trillion invested globally in research every year” and “transforming research cultures and systems.” Since 2022, he has also been a member of the International Science Council (ISC).

During his visit to Brazil, Wilsdon gave an interview to Agência FAPESP. Read the main excerpts below.

Agência FAPESP – In your presentation, you mentioned terms such as “metascience,” “science of science,” and “research on research.” All of these are somehow related to the study of how research is conducted, published, and evaluated. Can we consider them synonyms? What exactly do they refer to?
James Wilsdon – It’s a little confusing, I know. There are four terms that have become more prominent in the last decade: “metascience,” “metaresearch,” “science of science,” and “research on research.” None of them are entirely new. They are fields of research that have existed for over a hundred years in different forms. But they all represent a growing appetite, a growing enthusiasm for using robust scientific methods to analyze the scientific system itself. So, the best definition would be: metascience is the science of science. It’s using a set of scientific methods to understand how the scientific system works. And using that knowledge and the evidence obtained to intervene in the system, to try to improve it and make it work more efficiently and fairly. So, all these terms represent the use of more sophisticated methodologies to understand and change science.

Agência FAPESP – The Research on Research Institute was created in 2019, and several initiatives have since emerged in this field, such as the Metascience Unit, created by the British government in 2024, and more recently, the Metascience Alliance. Why do you think this topic has gained more relevance in recent years?
Wilsdon – Well, yes, there are many things happening in terms of trying to bring a little more coordination to this field of work. There are many more researchers getting involved with these issues [bottom-up initiatives]. They continue to do biomedical research or research in psychology, economics, physics, or whatever they do, but they devote part of their time to researching research. Studying how the scientific system itself operates. And there are also government [top-down] initiatives, such as the Metascience Unit. In this case, the central government and the national funding agency have come together to try to accelerate this type of work in the UK. And there are similar things happening in several countries. There’s also a movement on the part of research funders, whether public or private foundations, to support initiatives in this area. One example is the Welcome Trust in the UK, which is investing in OpenAlex, a database of publications and citations. As it’s an open database, anyone can use it without a license to do metascience.

Agência FAPESP – In an editorial published on July 8, Nature magazine stated that “metascience can improve science, but it also needs to be useful to society.” How can we show the importance of this field of study to those outside academia?
Wilsdon – That’s an important challenge. Metascience aims to prevent the loss of resources and increase the impact of research. And, of course, the more efficient use of public money benefits society as a whole. But it’s something that’s a little distant from people. It’s harder to “sell” than the idea of “doing research to cure cancer.” But if we want to do cancer research, it's important to study how to allocate resources efficiently and avoid duplication [overlapping efforts]. In some areas, there are many examples of poor research practices, studies that cannot be replicated, non-reproducible results, and, in extreme cases, scientific fraud of various kinds. So another goal of metascience is to draw attention to things that aren’t working well, so that we can fix them. In my view, all of this has a public benefit. However, the Nature editorial is correct: it’s a challenge for the metascience community to try to demonstrate more visibly the value of all this to a broad group of stakeholders and to ensure that we’re focusing on the most important social outcomes, that we aren’t just doing this to play with science but to try to ensure that science can contribute with its full potential to the economy, society, health, the climate crisis, and all the big things we need to solve.

Agência FAPESP – Is there a risk that any problems identified in the way research is being conducted will be used as justification for funding cuts?
Wilsdon – Yes, there’s a risk. We saw this happen in the United States recently [in May, President Donald Trump issued an Executive Order entitled “Restoring the Gold Standard in Science”, in which he stated that research in science, technology, engineering, and mathematics is facing a “reproducibility crisis” resulting from “data falsification” by researchers and the publication of “highly misleading” research by federal agencies. He also criticized diversity, equity, and inclusion policies that have “further politicized science”]. I think it’s wrong to blame the movement to improve science for the actions of Donald Trump and his administration, because I don’t think they’re acting in good faith, but rather looking for an excuse. But, of course, we have to be aware that this kind of thing can happen. But what’s the alternative? If we’re aware that there are problems in science, should we just sweep them under the rug? In the long run, this will be worse for science, because eventually the facts will come out. We don’t really have an alternative. We need to be smart about how we present the facts. Among the people I know who are involved in metascience, open science, or reproducibility movements, no one advocates cuts in public investment in science – quite the opposite. Most of these people are also strong advocates of increased funding. What we want is for these resources to be spent wisely and efficiently.

Agência FAPESP – In your presentation, you mentioned experiments being conducted by the Research on Research Institute to test alternative methods to peer review in the evaluation of research projects. Could you explain what is being tested?
Wilsdon – Something we’ve been doing recently that’s attracted a lot of attention is what we call distributed peer review (DPR). It works like this: when you submit a funding application, you automatically agree to act as a reviewer for three or four proposals submitted under the same call. So everyone who submits a project also automatically becomes a reviewer. Why is this potentially useful? Because it solves some of the problems we see in the peer review system. The main one is the difficulty of finding high-quality expert reviewers. And even when one agrees to do the work, they often don’t deliver or are late. So when you’re organizing a call for proposals, the process ends up taking a long time. Evidence from early trials with the DPR model indicates that it dramatically increases the efficiency of the peer review process. Since all reviewers are also applying for funding, they have an interest in seeing the process move quickly. We conducted an initial test in a call launched by the Volkswagen Foundation in Germany, with positive results. The British government repeated this experiment in a call launched by UKRI [the UK’s research funding agency]. The results show that DPR halves the time between project submission and the final decision. Now, other funding agencies in Europe intend to try the experiment, including the Swiss National Science Foundation. It isn’t complicated, expensive, or difficult. It’s just a small adjustment to the process that has the potential to cut the time in half. These are small things that can bring big gains in terms of process efficiency.

Agência FAPESP – Isn’t there a risk that reviewers, in this context, will evaluate their own competitors negatively just to favor their own proposals?
Wilsdon – Potentially, yes, but it’s very evident, very obvious. Experiments have shown that, in general, people don’t behave badly. They’re behaving as reviewers and treating the proposals they evaluate the way they want their proposals to be treated. It isn’t a perfect method or one that works for all types of calls. Sometimes reviewers need to come from a different field or even a different sector, such as industry. But for classic academic calls, it has some potential for application.

Agência FAPESP – In addition to delays, are there other types of problems with the peer review system that this new method helps to solve? Evaluation biases, for example?
Wilsdon – DPR solves two problems: finding qualified reviewers and reducing project evaluation time. As for bias, the new method doesn’t necessarily solve it, but in theory it shouldn’t make it worse.

Agência FAPESP – The Research on Research Institute hosted the Metascience 2025 Conference this year, during which the Metascience Alliance was launched. Could you explain what this initiative is about?
Wilsdon – Basically, it’s an alliance of institutions, not individuals. There are research groups linked to universities or institutes, funding agencies, government ministries, among others. The Research on Research Institute is an important member. It’s an area that’s been very active, with a lot going on, and the idea behind this alliance is to try to coordinate efforts, connect and build the community in the best way possible. The initiative has been coordinated by the Center for Open Science in the United States, and I believe that between 25 and 30 organizations joined in the first week. It’s open to other interested organizations. Participation is free of charge, initially. We have funding for about two years of work. Eventually we’ll need more, but it won’t be anything major. This initial funding comes from the Center for Open Science, and we raised a little more during the Metascience 2025 Conference. It’s just an attempt to strengthen this field, bring more coherence and cohesion. And it’s really open, whether to public or private entities.