p-Hacking, Retractions and the Dark Side of Research

    07/09/2014

    Science is a noble human endeavor, but like all human endeavors it is subject to human error and vice. While most of us try to maintain the strictest standards in experimental design, statistical data analysis and a rigorous peer review process before papers are published, some will inevitably game the system in order to get their papers out faster and rise through the academic or corporate ladder. The lack of requirement by certain journals when it comes to reviewing raw data and the large amount of papers submitted for publication in the top Nature, Cell and Science magazines serve to exacerbate a growing trend in data falsifications.

    A study by the University of Pennsylvania in 2012 highlighted the problem of "p-hacking" in research. P-value is seen as the standard measure for statistical significance, required as a minimum by all journals to show that data between experimental and control groups really differ strongly. However, when researchers do not see a clear difference, they sometimes manipulate the data by cherry picking experimental samples which happen to differ more from control samples. The resulting p-value would decrease and the difference becomes significant.

    Listen to NPR's story about p-Hacking

    Re-analyzing data if no significant difference can be found is valid in certain cases (for example re-defining the paradigm when counting the shapes of different neurons in a big assay). But the nefarious practice of personally changing data is misleading and its prevalence in the world of academic publications is troubling. What makes it sad is that this has been happening for many years, as shown by a famous reformer, John Ioannidis from Stanford University, who published a paper called "Why Most Published Research Findings Are False" in Plos Medicine in 2005.

    Of course when fraudulent data is found and papers get retracted, serious consequences can result. The suicides of Yoshiki Sasai (RIKEN institute) this year and Yu-yi Lin (formerly of Johns Hopkins) last year embody the worst examples of how scientists fall from grace. In their cases, it was not merely a case of p-Hacking but a series of fraudulent figures and images that led to the retractions.

    Yoshiki Sasai, professor of regenerative medicine at the Riken Institute. (Picture from Nature Magazine).

    This year alone, I recount a whole slew of stem cell research papers that have been retracted from top tier journals, suggestive of systemic problems with the lack of standards in this field. In fact, an entire industry has erupted to follow a growing number of retractions in academic journals, with websites like retractionwatch.com leading the charge. Even the White House is taking notice of the lack of reproducibility in published scientific data.

    Publishing false data to fan the flames of your novel idea is not new and is not restricted solely to academia. Indeed such acts in other sectors, such as clinical research trials or business analysis, can be far more harmful to the world at large rather than to the reputation of few professors. But as scientists, if we are to move forward with publicly funded programs and generous philanthropic donations it is our duty and responsibility to uphold the utmost sincerity when it comes to publishing true data.

    Retraction Watch website: