Amaral 2021 Nature
|Amaral OB, Neves K (2021) Reproducibility: expect less of the scientific paper. Nature 597:329-31. https://doi.org/10.1038/d41586-021-02486-7
Abstract: Make science more reliable by placing the burden of replicability on the community, not on individual laboratories.
• Bioblast editor: Gnaiger E
- The Reproducibility Project: Cancer Biology has so far managed to replicate the main findings in only 5 of 17 highly cited articles3, and a replication of 21 social-sciences articles in Science and Nature had a success rate of between 57 and 67% (ref. 4).
- We do think that the effort will pay off in terms of reproducibility. But if every paper in discovery science is to adopt this mindset, a typical high-profile article might easily take an entire decade of work, as well as a huge budget.
- There are typically three main expectations for a top-notch article in laboratory science: first, report original findings from exploratory research; second, confirm that they represent robust phenomena through further experiments using different methods; and, finally, suggest theoretical mechanisms to explain the results. However, these represent different aspects of the scientific process and do not have to be achieved all at once5.
- Studies have shown that neither statistical power6 nor quality of reporting of individual experiments7 improve as journal impact increases.
- Articles byindividual research groups should thus be regarded as preliminary by default.
- Basic exploratory science would be helped if editorial policies reduced requests for new experiments and refrained from demanding evidence of clinical potential.
- There needs to be training, funding and rewards for researchers to focus on managing collaborations, participating in large experiments and synthesizing data — especially because this involves sacrificing academic freedom to some extent.
- Requiring researchers to register how they will blind their study is more flexible than enforcing how they do it, but still serves to eliminate bias.
- Moving the burden of reproducibility from individual researchers to organized communities can ultimately raise the bar of what is considered scientific fact, and could also have a salutary effect on the public communication of science.