A long effort to reproduce 100 studies that had been published in three leading psychology journals found that more than half of the findings did not hold up when retested. The analysis was done by research psychologists and their conclusions have confirmed fears that research into social sciences like psychology may need to be re-evaluated.
There was neither evidence of fraud nor any evidence that the original studies were definitively false. Rather, the new research concluded that the evidence for most published findings was not nearly as strong as originally claimed. Very few of the replicated studies contradicted the original ones; the results were simply weaker.
The studies reproduced included important works on the dynamics of personality, relationships, learning and memory. Therapists and educators use this research to help make clinical decisions. More than 60 of the 100 studies did not hold up.
Among the studies was one on mate preference that was widely reported. It had found that attached women were more likely to rate the attractiveness of single men highly when the women were highly fertile, compared with when they were less so. In the reproduced studies, researchers found weaker effects.
The project began in 2011 at the University of Virginia. More than 250 researchers volunteered to rigorously re-perform 100 experiments in close collaboration with the original authors.
The fact that so many of the results could not be replicated fully is troubling. Some place blame on the rush to publish. Many areas of science operate under "publish or perish," which means that researchers must publish a paper in a science journal on a regular basis. There is also a bias that favors new and flashy results and gives little or no incentive for replicating the work of others.
The new analysis focused on studies published in three top journals: Psychological Science, the Journal of Personality and Social Psychology, and the Journal of Experimental Psychology: Learning, Memory, and Cognition.
The project's authors write that despite the painstaking effort to duplicate the original research, there could be differences in the design or context of the reproduced work that account for the different findings.
Several psychology groups came out in support of the new research, noting that the people who redid the research were also psychologists. The results of the project were published in the journal Science. You can read the abstract here.
