What hurts science - rejection of good or acceptance of bad?
Yesterday, Science published a story
by John Bohannon about acceptance of a fake and deeply-flawed paper at open access journals, despite peer review. Disturbingly, 157 journals accepted the bogus article and only 98 rejected it. Scientists and some journalists swiftly pointed out the grave problems with this attack on open access - this sting operation highlights problems with traditional peer review, but it says very little about open access, as the same experiment was not performed on subscription journals ((Great list of the responses here
To me, the stunning part of this is that a journal with the title "SCIENCE" published a fake study, without a control, about the problem of accepting fake studies. But the bigger question is - how much damage is there from publication of poor science? Does anyone really read the journals that accepted this?
I took the 157 journals that accepted Bohannon's fake paper and asked how many articles are in the libraries of the PubChase users from them. The answer is that out of over 75,000 articles of our users, only 5 are from this set (all five are from the single journal Bioinformation
). In contrast, our users have 1,631 articles from 12 of the 98 journals that rejected the paper ((Of the 98 journals that rejected the fake paper, PubChase users have articles from 12 of them: PLOS One, mBio, Neurosurgical focus, International journal of biological sciences, Chinese medical journal, American journal of nuclear medicine and molecular imaging, Carcinogenesis, Yonsei medical journal, Current issues in molecular biology, Anti-cancer drugs, Immunome research, Environmental health perspectives.)).
The real problem in science is not that bad papers get published; that has always been and will continue to be the case. The real problem is that good and important papers are rejected and delayed from publication by journals such as Science.
These delays hurt the progress of science and they demoralize and ruin careers
Finally, when it comes to publishing bad research, Science is not the journal that should be pointing fingers. The 2011 editorial "Retracted Science and the Retraction Index"
showed unambiguously that the higher the journal's impact factor, the higher its retraction rate. Not surprisingly, Science had the second worst retraction rate of all the journals considered in that editorial.