Amaral 2021 Nature

Amaral OB, Neves K (2021) Reproducibility: expect less of the scientific paper. Nature 597:329-31. https://doi.org/10.1038/d41586-021-02486-7

Abstract: Make science more reliable by placing the burden of replicability on the community, not on individual laboratories.

• Bioblast editor: Gnaiger E

Selected quotations

The Reproducibility Project: Cancer Biology has so far managed to replicate the main findings in only 5 of 17 highly cited articles3, and a replication of 21 social-sciences articles in Science and Nature had a success rate of between 57 and 67% (ref. 4).

We do think that the effort will pay off in terms of reproducibility. But if every paper in discovery science is to adopt this mindset, a typical high-profile article might easily take an entire decade of work, as well as a huge budget.

There are typically three main expectations for a top-notch article in laboratory science: first, report original findings from exploratory research; second, confirm that they represent robust phenomena through further experiments using different methods; and, finally, suggest theoretical mechanisms to explain the results. However, these represent different aspects of the scientific process and do not have to be achieved all at once5.

Studies have shown that neither statistical power6 nor quality of reporting of individual experiments7 improve as journal impact increases.

Articles byindividual research groups should thus be regarded as preliminary by default.

Basic exploratory science would be helped if editorial policies reduced requests for new experiments and refrained from demanding evidence of clinical potential.

There needs to be training, funding and rewards for researchers to focus on managing collaborations, participating in large experiments and synthesizing data — especially because this involves sacrificing academic freedom to some extent.

Requiring researchers to register how they will blind their study is more flexible than enforcing how they do it, but still serves to eliminate bias.

Moving the burden of reproducibility from individual researchers to organized communities can ultimately raise the bar of what is considered scientific fact, and could also have a salutary effect on the public communication of science.

Gnaiger E (2021) Beyond counting papers – a mission and vision for scientific publication. Bioenerg Commun 2021.5. https://doi:10.26124/BEC:2021-0005 - Living Communication

Labels:

BEC2021.5f, MitoFit2022QC