Daniel BuenoInterested in verifying the consistency of social psychology research studies, a task force of 100 researchers from a mix of countries attempted to replicate the findings of 27 studies that had recently been published in that field. In at least ten cases – that is, more than one-third of the total – the results were not confirmed. In another five articles, either the observed effect was smaller than that described in the original study or statistical problems were encountered that had not been reported by the authors.
“Replication helps us make sure what we think is true really is true,” Brent Donnellan, a psychologist at Michigan State University and one of the leaders of the task force, explained to the magazine Science. The irreproducibility of research findings represents a constant headache for the scientific community because it hampers the advance of knowledge and in some cases may reflect fraud.
The replicators cautioned that it should not be claimed that scientific misconduct took place. One of the studies in which the results failed to replicate suggests that physical events related to cleanliness, like hand-washing, can influence people’s behavior and momentarily prompt them to be less harsh in their moral judgments. Published in 2008 in Psychological Science, the paper in question endeavored to complement similar studies, one of which had shown that feelings of disgust could render a person’s judgments harsher.
“I feel like a criminal suspect who has no right to a defense,” said Simone Schnall, author of the article and a psychologist at Cambridge University in England. “I stand by my methods and my findings and have nothing to hide.”
Schnall argues that the methodology used to replicate her study differed from the original, that is, the replicators discarded some data that diverged notably from the average observations. Donnellan confirms this methodological difference but says that a number of strategies were used to avoid any distortion – yet in no case were Schnall’s findings confirmed. Donnellan says that the problem with the study was the limited number of cases analyzed by the researcher, which can produce a false positive result.
To reduce the impact on professional reputations, Daniel Kahneman, professor at Princeton University, has suggested the adoption of a “replication etiquette,” where replicators would make a good-faith effort to engage the collaboration of the original authors and the latter, in turn, would participate actively in replication efforts. At any rate, Kahneman points out, researchers who are responsible for studies that achieve spectacular results should try to replicate their findings before a colleague does.
Republish