A group of researchers from Stanford and the University of California, Davis, in the USA have proposed a new strategy to prevent errors and biases in psychology research findings, suggesting that the article review process should contain a specific step for verifying the statistical robustness of the results. The paper, authored by a team led by psychologist Tom Hardwicke of the Stanford Center for Innovation and Metaphysics, published in the journal Advances in Methods and Practices in Psychological Science, notes that while statistical reviews have been standard practice among biomedical journals since the 1970s, they are still rarely used by psychology journals. According to the group, the process could help address what has come to be known as the “reproducibility crisis” in psychology: a succession of scientific articles that have fallen into disrepute because their results were not confirmable in subsequent experiments—some due to errors, and others due to fraud not detected in the peer review process. “Serious concerns about the credibility of psychological research have been raised, and the misunderstanding and misuse of statistical methods have been implicated as an important cause,” the authors wrote.
Hardwicke interviewed 39 editors of psychology publications and found that 71% did not differentiate between traditional peer review, which assesses whether methods are appropriate and results are related, and a statistical review, which is capable of identifying mistakes in calculations and analyses or the use of samples too small to support categorical conclusions. He also observed that 44% of the editors consider this additional step unnecessary. There are exceptions, such as the journal Psychological Science, which since 2016 has been using a team of six statistical experts to evaluate papers submitted for review. According to the group, a statistical review would not solve all the problems currently faced by psychology researchers, but in combination with improved statistics training for early-career scientists, it could help alleviate the reproducibility crisis.
Republish