The race to find responses to the pandemic has put the preprint model—where scientific papers are shared on public servers before they undergo peer review—to the test. Recently published studies and data shed some light on the growing use and quality of this approach to scientific communication during the global health emergency. Researchers at the University of Zagreb in Croatia analyzed over 6,000 papers on COVID-19 uploaded to the bioRxiv and medRxiv preprint servers between January and September 2020. They focused on the subset of these articles that were later subjected to peer reviews and published in scientific journals. They observed that preprints related to COVID-19 were accepted for publication approximately 50 days after submission, while the delay exceeded 100 days for manuscripts on other research topics, reflecting the desire among journals to expedite the dissemination of knowledge about the pandemic.
Another study, described in PLOS Biology in February, analyzed the content of preprints and concluded that for those later published in journals, peer review had only a small impact, reflecting that even in their preliminary versions, most of them were already reasonably complete. Written by researchers from Germany, the UK, and the USA, the paper analyzed the abstracts of preprints on various topics shared on bioRxiv and medRxiv and published in journals at the beginning of the pandemic, prior to April 30, 2020. Of the articles on COVID-19, only 17.2% were changed between the draft and the peer-reviewed version. And most of these changes did not have any qualitative effect on the conclusions. Among preprints on other subjects, the number that underwent changes was even lower, at 7.2% of the total. The researchers did not analyze the content of papers that were never published in journals.
Of course, there have been occasions during the pandemic when low-quality or even fraudulent manuscripts have been shared on preprint servers. On January 31, 2020, a group of researchers from India uploaded a paper to bioRxiv suggesting that there are similarities between proteins from SARS-CoV-2 and HIV, the virus that causes AIDS. The paper had widespread repercussions on Twitter, fueling conspiracy theories about the origin of the novel coronavirus. After the study was scrutinized and refuted by other scientists, the article was quickly removed by its authors on February 2.
Preprints, however, were not necessarily the biggest source of false research data. By mid-February this year, the website Retraction Watch had identified 212 scientific articles on COVID-19 that were subsequently retracted due to errors, methodological problems, or fraud. Only 43 of them were preprints. Most of the list—169 of the total—were published in journals and had undergone peer review. Even prestigious journals such as The Lancet and The New England Journal of Medicine retracted articles in the face of evidence that the results were based on fraudulent data provided by a company called Surgisphere, which had no legitimate means of obtaining the data. Among the retracted COVID-19 articles, there were also examples of clear failures in the review process, such as an article published in the Journal of Biological Regulators and Homeostatic Agents that claimed that 5G cell phone technology can induce the novel coronavirus in epithelial cells.
Olavo Amaral, a doctor from the Institute of Medical Biochemistry at the Federal University of Rio de Janeiro (UFRJ), published a paper in the journal Research Integrity and Peer Review in 2020 comparing the quality of preprints and peer-reviewed articles in the biomedical literature—the sample under analysis was taken from 2016, before the pandemic. The study showed that peer review is capable of incrementally improving article content by correcting formal errors and small omissions. The quality of the preprints, however, was only slightly lower than the peer-reviewed articles, supporting the idea that the former should be considered valid scientific contributions. “The most obvious advantage of preprints is the speed with which they are disseminated. Delaying the communication of research results for months while the peer-review process is carried out is unacceptable for findings that can impact people’s health,” he says.
Amaral is head of the Brazilian Replication Initiative, a project aiming to repeat 100 experiments from Brazilian biomedical articles to see if it is possible to replicate the published results. He does not see preprints as any more likely to contain data that cannot be confirmed later. “It’s not true that peer review guarantees a certain level of quality. Reviewers are only able to assess the available data and suggest improvement. It’s difficult to detect if an author lies or uses false data, which is commonly the case with retracted articles,” he says.
Abel Packer, director of the open-access journal library SciELO Brasil, says that peer review is a pillar of scientific communication and the adoption of preprints as a publication standard is far from reaching a consensus. “In the current model, the responsibility for published content lies with the journal, editors, and reviewers. With preprints, it all comes down to the author. There is resistance to that,” he explains. He acknowledges that the instant publication of preprints has gained more acceptance during the pandemic. “Today, when the findings of a preprint are reported, there is always a warning that the result is yet to be reviewed by other scientists. The public now have a better understanding of how science works.”
Packer, who also coordinates the open-access repository SciELO Preprints, says that it is possible to prevent poor quality research from being shared. On the Brazilian platform, all manuscripts are analyzed by a moderator before they are published to ensure they meet certain minimum requirements, such as approval by an ethics committee when necessary and a description of the methods used. Authors also need to identify themselves via the Orcid system, making it possible to assess whether they have published any previous scientific work. “If an author is young and has never published anything before, we will only share the manuscript on the platform if it is endorsed by a recognized scientist,” he says. During the pandemic, several scientific journals, such as The Lancet and JAMA, adopted a middle-ground approach, creating their own preprint servers to quickly publish important results while the peer-review process was ongoing. Journals in the SciELO collection adopted the same measures. Upon receiving new subsmissions, editors recommend that the authors also upload them to the SciELO Preprints server. “This way, the knowledge is already available while the paper is being reviewed,” says Packer.
Republish