Infodemics, research quality, open science and secondary sources of research
By Edoardo Corsi Decenti
The opinions expressed in this interview are those of the participants and not necessarily belong to the institutions the participants are affiliated with.
During the 15th European Public Health Conference, I had the opportunity to interview Lazaros Belbasis, a 31-year-old Greek physician – scientist, who currently works as an early career research fellow at the Nuffield Department of Population Health at University of Oxford. Lazaros participated in the workshop “Quality of COVID-19 science: meta-research and the ethical implications for public health”, where he gave an interesting presentation on the quality of COVID-19 research. He talked about an overview of the articles assessing the quality of COVID-19 literature, focusing on the key findings from meta-research work on COVID-19. He also presented two case studies he conducted on the quality of COVID-19 research. The first one was a field-wide systematic review and meta-analysis on prognostic factors for adverse clinical outcomes in patients with COVID-19 [1]. He and his colleagues reported a high risk of bias in at least one domain in 99% of the 428 eligible articles, which was mainly attributed to poor reporting of study setting and statistical analysis. He also emphasized on the phenomenon of data dredging and “salami slicing” observed in COVID-19 research, indicating the fact that researchers often were focusing more on the quantity that the quality of publications for COVID-19. The second one focused on the quality of articles estimating the excess mortality during the pandemic. His main observation was that many articles used inappropriate methods to model the expected mortality and often did not calculate excess mortality estimates for different population strata.
After his presentation, Lazaros and I had a vivid discussion about infodemics, research quality, open science and secondary sources of research. I present the main points of our discussion in the following paragraphs.
“What’s an infodemic?”
In the beginning of the interview, Lazaros and I discussed the issue of “infodemic”, a phenomenon referring the rapid dissemination of mostly inaccurate and misleading information during a disease outbreak and is well known among health policymakers and health professionals in general who dealt with the COVID-19 pandemic.
“Dissemination of research findings to the public is important, especially during a pandemic. Both scientists and mass media are responsible for delivering information from reliable and high-quality research to the general population. However, during the pandemic, we observed a rapid dissemination of inaccurate information about COVID-19. For example, a research study assessed the quality of the top 250 according to Altmetric Attention Score, a metric that quantifies the attention a research article has received in mass media, social media and other electronic sources [2]. This study showed that the majority of the COVID-19 studies that received high attention were of low quality and presented poor adherence to reporting checklists.”
“What can be done to fight an infodemic?”
After discussing about COVID-19 infodemic, I asked Lazaros what could be done to avoid other infodemics in the future. Lazaros highlighted the importance of (a) training the scientists and journalists in the effective dissemination of research findings and (b) giving voice to experts in the decision-making process and the management of emergency scenarios.
“To fight an infodemic, first of all, we need high-quality research, which can be achieved by better training of future scientists, by rewarding good research practices and by promoting international collaborations between scientists. We should always be aware that poor quality research can hurt the trust of people to the scientific findings and the scientists. Also, we need responsible dissemination of research findings to the public. Researchers should be trained in how to present their findings to lay people and they should remember that engaging the public is part of their job. Moreover, journalists that are responsible for communicating the research findings to their readers or audience should have appropriate scientific training and realize that presenting findings from poor quality research can mislead the public.”
“What do you think about the phenomenon of publishing poor quality papers?”
The next topic we discussed about was the reasons of the poor quality of scientific literature.
“According to the famous aphorism “publish or perish”, researchers have the pressure to publish a lot of articles, and this usually affects the research quality. Especially during the pandemic, we observed a reduction in the publication time, which was accompanied by a reduction in the quality of the published research, and this observation is supported by various meta-epidemiological studies. The current structure of the academic system is primarily responsible for this situation. When scientists apply for promotion or research funding, they are judged mainly by their scientific productivity, their citations and the impact factor of journals where their research has been published. To improve this situation, good research practices, such as open science, data sharing and scientific collaborations, should be appreciated and rewarded by the academic system. ”
“And what do you think about open access?”
“Open access represents a unique opportunity for spreading science, especially for countries and institutions that can’t afford paying journal subscriptions. At this point, I would like to emphasize the fact that open access publications are only one component of open science. Data sharing and code sharing improve the quality of science, make the scientific process transparent, and facilitate the replication of scientific findings and scientific collaborations.”
Returning to the concept of systematic reviews, I would like to ask “what is the role of secondary sources such as systematic reviews in creating scientific evidence?”
“Systematic reviews are the cornerstone of evidence-based medicine and represent the best approach to summarize and assess the scientific evidence. The rapid scientific production during the pandemic highlighted the importance of living systematic reviews, which are continuously updated systematic reviews that aim to provide an up-to-date overview of scientific evidence. Living systematic reviews constitute a great example of high-quality collaborative science during the pandemic. However, they require a considerable amount of resources in order to remain up-to-date. This fact highlights the importance of allocating research funding for high-quality systematic reviews and meta-analyses, especially during natural disasters.”
Written by Edoardo Corsi Decenti, teaching research methodology at the Saint Camillus International University of Health and Medical Sciences, Rome, Italy, and EUPHAnxt fellow at the 15th European Public Health Conference, held 9-12 November 2022 in Berlin, Germany.
References
[1] Bellou V et al. Prognostic factors for adverse outcomes in patients with COVID-19: a field-wide systematic review and meta-analysis. European Respiratory Journal 2022;59:2002964
[2] Khatter A et al. Is rapid scientific publication also high quality? Bibliometric analysis of highly disseminated COVID-19 research papers. Learned Publishing 2021;34(4):568-577
Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or HaDEA. Neither the European Union nor the granting authority can be held responsible for them.
