Research funding agencies discuss criteria for project selection and evaluation of results

The proposal is to find a balance in funding between research that delivers societal and economic impacts and research that advances knowledge.

By Elton Alisson & Maria Fernanda Ziegler | Agência FAPESP – The growing expectation among different sectors of society that publicly funded research should have a societal and economic impact is not a new phenomenon. On the contrary, it is but the latest episode in a debate that has been going on ever since science began, said Peter Strohschneider, President of the German Research Foundation (DFG).

At the Annual Meeting of the Global Research Council (GRC) in São Paulo, research funding agencies agreed they should respond to growing demands for more impactful research in two ways. One is to introduce evaluation of the economic and societal impact of research into the criteria for project selection. The other is to pursue more effective assessment and communication of the impact of publicly funded research.

“These two approaches are complementary, but the rationale for each one is different,” Strohschneider said on opening the GRC Annual Meeting on May 2. This is the first time the event has been held in Brazil. Ending May 3, it was organized by FAPESP, DFG and Argentina’s National Scientific and Technical Research Council (CONICET). Fifty heads of research funding organizations from as many countries on all five continents attended.

“Introducing the societal and economic impact of research as a funding criterion is based on the assumption that this impact can be maximized, while the need to better assess and demonstrate the impact of funded research results is based on the understanding that while the projects supported may have had the desired impact, their results need to be more visible to society,” Strohschneider said.

Both approaches, he added, can raise awareness of the contribution of research to society and the economy, thereby strengthening the arguments used by the agencies that belong to GRC to justify the use of public funds in support of research projects. However, they should think long and hard about how to conduct this evaluation in order not to penalize basic research.

When assessing projects, Strohschneider stressed, “it’s important to consider not only the societal and economic returns but also the value added of curiosity-driven research that transcends the knowledge frontier as essential to fostering the development of vibrant national research ecosystems”.

“We must find a balance in our funding efforts between research that delivers societal and economic impacts and research that advances knowledge,” he said.

For Marco Antonio Zago, President of FAPESP, assessment and demonstration of the societal and economic impacts of research as a funding criterion is a controversial matter on which a complete consensus cannot be expected, although there are some points on which all can agree. One is the need to respond positively to growing pressure to demonstrate that the research projects supported by funding organizations benefit society, especially because they are publicly funded.

“How each agency complies with this rule will vary,” he said. “Ultimately, however, societal, economic and regional impacts will always influence the types of research any agency supports, as a criterion for selecting projects or as general guidelines and policies that create specific research programmes or modify short-term projects and long-term priorities.”

“Our mission as heads of research funding agencies is to facilitate research,” Zago continued. “To do so we must strive to ensure optimal use of the funds provided by society, so that every project we support is of the highest scientific quality and is highly likely to contribute to the expansion of scientific knowledge or to have a positive impact on society.”

Scientific merit

The potential impact of research is considered for the purposes of project selection in the United States. Since 1997 the National Science Foundation (NSF) has used only two funding criteria: intellectual merit, i.e. the potential to advance knowledge; and broader impact, i.e. the potential to benefit society.

According to NSF Director France Córdova, the institution receives some 50,000 funding applications each year. “There are many more excellent proposals than funds to support them,” she said. “Last year alone we were unable to award some US$4 billion to support excellent projects. That’s the amount of funding requested by the proposals we couldn’t approve even though they were rated very good or higher.”

Córdova explained that the scientific merit criterion helps select proposals at the knowledge frontier, citing as an example the research conducted by Frances Arnold, who won her first NSF grant in 1988 and was awarded the 2018 Nobel Prize in Chemistry. “Arnold designed a method called directed evolution, in which rounds of mutation and selection are used to improve the efficiency of enzymes towards a desired function,” Córdova said.

Years after that first funded project, Arnold’s work has made possible the development of novel pharmaceuticals, biofuels, and processes to make industrial chemicals without toxic metals or organic solvents.

“You may be wondering how a researcher can know on submitting a research proposal in basic science what the project’s outcome will be? Of course, they don’t know. That’s why the definition of intellectual merit and broader impact focuses on the potential to advance knowledge and benefit society,” Córdova said.

The NSF view of societal and economic impact extends to formal or informal educational outreach as well as high-risk research, which often involves partnership with industry, other agencies and international sponsors to build capacity and leverage resources.

“NSF’s support for high-risk research has produced some surprising discoveries,” Córdova said. “Our seed funding helped advance the internet, solar panels and life-saving medical drugs. We’ve contributed to progress in understanding the brain, curing disease, and the development of 3D printing.”

Interdisciplinarity and engagement

For Mark Ferguson, Director-General of Science Foundation Ireland (SFI), what matters most is to ensure that funders choose research projects wisely. “There has to be interdisciplinarity, and it’s necessary to encourage engagement with people. However, it’s important to realize there isn’t just one model. You can’t fund disruptive research by issuing a conventional call for proposals,” he said.

The idea that societal impact should be one of the criteria for selecting research projects is not yet a consensus. In Austria, for example, the main goal is to stimulate transdisciplinary and transformative research. “For this reason we’re no longer using societal and economic impact as a research funding criterion. We’re funding primarily curiosity-driven and interdisciplinary research projects encompassing the humanities all the way to chemistry and statistics,” said Klement Tockner, President of the Austrian Science Fund (FWF).

There is evidence that focusing too early on societal and economic impact may restrict innovation and risk taking, Tockner stressed. “Austria is interested in promoting diversity not just in terms of disciplines and topics, but also in terms of research projects. This is important if we’re going to focus on the interdisciplinary and transdisciplinary future of research based on diversity,” he said.

Austria has created alternative ways of facilitating links between research, society and the economy while stimulating cross-cutting and disruptive research in future. “We want to have a perspective on how to identify research areas in which new topics are emerging and how to communicate their use to society,” Tockner said.

Another alternative is the creation of Pioneer Labs at the end of this year as an incubator for transdisciplinary and transformational research projects. “We’re concerned about societal impact but we believe our prime responsibility is to make sure research reaches the highest stage of quality and excellence,” Tockner said.

According to Tockner, Austria has another funding organization that focuses on applied research and economic impact. “They have more funds than we do,” he said.

Patricia Ellen, São Paulo State Secretary for Economic Development, Science and Technology, also attended the opening of the event. According to Ellen, more emphasis should be placed on assessing the societal and economic impact of research.

“In recent decades science and technology have played a key role in driving many improvements to our society, such as longer life expectancy, urbanization and rising per capita income. But we’re currently addressing many challenges, and research will be critical in helping us find solutions,” she said.

More information on the 8th Annual Meeting of the Global Research Council: www.fapesp.br/eventos/grc.