The work of Dutch microbiologist Elisabeth Bik has become a benchmark in identifying evidence of errors and fraud in scientific articles, especially duplicated or manipulated images. She estimates she has analyzed more than 200,000 studies, 5,500 of which had problems. The researcher writes about them on her blog Science Integrity Digest, to which she has been dedicated full time since 2018, when she left her job at uBiome, a biotechnology company headquartered in California, in the United States. She also gives lectures on scientific misconduct and consults with publishers and teaching and research institutions.
Educated at the University of Utrecht, Bik works closely with online platforms such as Retraction Watch, which monitors and reports on retractions of scientific studies (see Pesquisa FAPESP issue nº 282), and PubPeer, where users discuss published papers and point out inconsistencies. The researcher was one of the first to point out problems in the methodology and findings in the controversial article by French physician Didier Raoult, regarding the use of the antimalarial drug hydroxychloroquine and the antibiotic azithromycin for treating Covid-19. She has since been attacked on social media and threatened with lawsuits by Raoult and his collaborators.
Bik spoke to Pesquisa FAPESP in an interview via Zoom at the end of October. From her home in San Francisco, she discussed her career as a researcher, the challenges of detecting falsified images in articles, and the efforts of publishers to combat these and other types of scientific misconduct, in addition to the controversies involving Raoult.
How did you become interested in identifying fraud in scientific research?
I attended an event on plagiarism in 2013 and when I got home, I decided to search some excerpts from an article of mine on Google Scholar. I expected to find similarities with things from my past, but I ended up finding snippets of my text in someone else’s more recent work. I thought it was bad luck, but then I found another study with segments from my articles. I began dedicating some time to it as a hobby. Sometime later I found a doctoral thesis with excerpts from an article I had published. The thesis also used the same western blot image [for detecting proteins] in different chapters to illustrate different things, one of which appeared upside down. I started to analyze other articles and realized two things: that I had a talent for finding duplicate images and that this is a recurrent problem.
How did you come to the decision to dedicate yourself exclusively to this work?
There came a time when I realized that I could contribute much more to science by working on scientific integrity than at my traditional job. My husband and I evaluated our financial situation and decided to try it for a year. If it didn’t work out, I would go back to my old job. In the end, I managed to make a go of it.
Is it difficult to identify image fraud when peer-reviewing studies?
It’s not always fraud. It may be an honest mistake. But, in general, I noticed that magazines and reviewers weren’t paying much attention to this. They are more concerned about plagiarism and methodological flaws in the papers. That’s why I decided to talk about these cases on my blog and on social media. The idea is to make people more aware of this problem and capable of identifying it.
Are cases of falsified or duplicated images on the rise?
I think they’re falling, but I don’t have numbers to support that. I know surveillance has increased and I frequently get emails from reviewers saying they’ve identified manipulated images. The biggest problem is research paper mills, which produce false images and incorporate them into fraudulent scientific articles. It’s very difficult to recognize them. They are unique images, created by artificial intelligence. It works like those sites that create faces of people that don’t exist using parts of real faces found on social media or in databases. Paper mills use several real images to assemble one single—and fake—image. You sense something is wrong but can’t determine what it is.
How big of a problem is this?
It’s very problematic because it involves organized crime. It’s a recurrent issue in China due to a single government requirement. To graduate from medical school, students need to publish a paper. But they aren’t interested, nor do they have the time to do research. So they buy fake articles. These papers look a lot like each other, as if they were written from the same form. There are thousands of them that have infiltrated the medical literature, to the extent that they’re undermining the credibility and reliability of all articles produced in China. Many editors are refusing to review manuscripts written by researchers attached to Chinese hospitals or medical schools.
In addition to paper mills, what are the most common problems involving fraud in scientific images?
There are three categories. The first is the issue of duplicated images. In other words, the same image is used to illustrate different things. The second involves identical and repeated images, but presented in different ways, such as upside down or inverted. The third category refers to duplications within the image, such as when the same cell appears twice, three times in the same figure.
How many articles have you studied?
About 200,000, and out of those around 5,500 had problems.
And how many were retracted?
At last count, 624.
Why so few?
Many are under analysis and might be retracted in the future. Some shouldn’t be retracted because they made honest errors and the authors are cooperating to resolve the issue. In those cases, they will just be corrected. Of the articles I reviewed, 509 have already been corrected.
Is there any resistance from editors to retract these articles?
Some are just overwhelmed with the number of suspect papers to review. I’ve had to email editors who had as many as 40 problematic articles. There are situations where the suspected article was authored by someone from the journal’s editorial board, or the editor is close to the author of the article. I notice in these cases that things develop more slowly.
How do suspect articles make their way to you?
I often get emails from people asking me to look at pictures in this or that article because they suspect they have problems. They also write to me asking to analyze studies by specific researchers or institutions. When I come across a suspect article, I look at the list of authors to see if it contains the name of someone who has signed problematic papers in the past. I try to follow various clues.
Is it possible to differentiate an intentional case of misconduct from an honest mistake by analyzing the images in an article?
It’s hard to believe that it’s an honest mistake when you see identical cells in the same image, as there’s no reasonable technical explanation for this. It’s different when it comes to duplicated images. Sometimes, you can tell by the caption that another image was supposed to be there. There may have been some kind of problem generating the file. Therefore, I tend not to make accusations when I refer these cases to the editors. I just write in and tell them that there seems to be something wrong.
Do you use software to analyze the manipulated images or do you do it manually?
I did everything by eye until a few years ago. These days I also use a software program called ImageTwin. It works fine for some images and can find similarities in seconds. But you can’t rely on it completely. The software has missed some quite obvious duplications, but it also identified some that I would never have found myself.
Image-analysis software still isn’t as good as the ones used for finding plagiarism, is it?
Plagiarism software also has limitations. The point is that it will always take a human being to analyze the results from these programs. There’s software that identifies when you use quotation marks in a citation to define a concept in an article, but if you put the same definition in italics, the program doesn’t recognize it.
Personal collection
Examples of duplications identified by Bik in an image in a scientific articlePersonal collectionAre publishers investing in improving image analysis software?
They are testing some options, but I don’t know what stage of development they’re at. I wanted to have access to these programs to test them out. I’ve already shared my data with people who claimed to be working on developing such software on the condition that they could use it to test their final product, but I haven’t heard back from them.
Could you explain the lawsuit Didier Raoult says he’s filing against you?
One of his collaborators announced in April on Twitter that Raoult had filed a lawsuit against me. It published a photo of a complaint he’d allegedly filed with a prosecutor in Marseilles, which contained my home address. He later deleted the post. Some newspapers have gone after the story, but as far as I know, so far he hasn’t filed a lawsuit. It appears to be a police report, in which he accuses me of harassment, blackmail, and extortion.
What are these accusations based on?
On comments I made about his studies on PubPeer. Every time someone comments on a study on the platform, the authors receive an email. I posted over 60 comments on Raoult’s articles, particularly the one involving hydroxychloroquine, so he received over 60 of my emails. In any case, I don’t believe this is a lawsuit; it looks more like a threat.
What problems did you identify in 2020 in his study on the use of hydroxychloroquine for Covid-19?
The study involved a small group of participants, and they did not randomize them into a control group and a treatment group. Nor were they from the same hospital. Some didn’t respond well to hydroxychloroquine. One died. Others were transferred to the ICU. As it turns out, these individuals were left out of the research and this, of course, influenced the results. As if that weren’t enough, the study began before being approved by the ethics committee. Raoult submitted the article to the International Journal of Antimicrobial Agents on March 16, 2020. It was accepted a day later and posted online on March 20th. This means that the peer review was done in just 24 hours. It was later discovered that one of the authors was the editor of the journal that published the article.
After this incident, you began getting attacked on Twitter. How has this affected your work?
It’s not pleasant to read cruel messages about yourself on the internet. If it were just the occasional person with a few followers insulting me, fine. The problem is that some profiles have thousands of followers. And they’re not just unknowns. There are also academics, like Eric Chabrière, who released my address. He’s a professor at the institute that Raoult is director of, and he published and shared several montages showing me behind bars, as well as posts with phrases like “You will be arrested if you come to Europe,” or “I will visit you in jail.” There are also recurring attempts to disqualify me as a scientist.
In what way?
They imply that I’m getting paid by pharmaceutical companies. They also say that I only have 40 articles published, while Raoult has 3,500.
Did he respond to your comments on PubPeer?
No. He prefers to attack me on YouTube, with texts in French and legal threats.
Thousands of fake articles produced by paper factories in China have infiltrated the medical literature
Is it common for researchers to react badly to your criticism?
This behavior is restricted to a specific type of scientist, who is abusive, used to doing things their way, who thinks they’re always right. The majority know how to deal with criticism and their own errors. No one likes to see their work challenged, but researchers I’ve had problems with could have refuted my considerations with data or documents. If they resorted to threats and insults, it’s because they had no arguments.
Many articles on Covid-19 have been published on preprint platforms without peer review. Were there many problems?
I don’t think this has increased the cases of misconduct. The main problem, as I see it, is that many papers on Covid-19 are bad. There are articles like “The impacts of Covid-19 on pediatric patients” or “The impacts of Covid-19 on cancer patients”, with weak or irrelevant data, but not necessarily involving misconduct.
Publishers and universities often invite you to lecture or provide consulting. What do you offer them?
In the lectures I talk more about the paper mills and how to identify fake articles. My consulting work consists of analyzing cases of articles suspected of containing falsified images. When a PubPeer user suggests that a work may have problems, the institution to which the author is connected—or the publishers—ask me to analyze the case and indicate whether the suspicions are well founded. These institutions also ask me to analyze cases that aren’t yet public.
What can the scientific community do to reduce cases of misconduct?
I think it should look more at what goes on in the labs. Many are still headed by abusive researchers who take advantage of their position to harass, insult, or threaten their subordinates. Many times young scientists engage in misconduct in order to meet their superior’s demands and save their jobs. I think institutions should listen more to young researchers.
Do you think your job could become a specialty in the future?
Yes. Some publishers are hiring people to scrutinize papers suspected of having problem images and coordinate quality control. I’ve also seen universities becoming interested in these types of professionals to review their researchers’ articles before they’re submitted for review.
I noticed that you use your Twitter profile to teach people to spot duplicate images.
I’m secretly training my followers.
