On the heels of the discrimination index manuscript outlined in our previous post, we also completed a manuscript about the prototype observatory designed to respond to published COVID-19 science, which was introduced in a post last year. We positioned this work in the broader context of assessing and improving the peer review system in scientific publication and research grant evaluation. The 5,600-word text is available as preprint and was submitted for peer review in a traditional scholarly journal.
To illustrate the functionality and potential value of the observatory (or rather the general idea of such an observatory), we picked five articles that were heavily critiqued after their publication. The above screenshots from the live repository are about the “Bangladesh mask study” by Abaluck et al. (2021). We include and discuss different types of responses, from direct responses as letters or comments in the journal of the original publication, including the original authors’ responses where available, to news or opinion pieces published in magazines and media, blog posts, and entirely new, scholarly articles written to correct the original article.
These often very well-argued and -articulated critiques are necessary in order to understand the validity of the original sources, yet this valuable material will be hard to come by for decision-makers and future researchers. Outside of encouraging the responding authors to publish their assessments in the scientific literature, collecting the information for easier access appears to be the second-best option in order to make them visible and promote them outside closed communities of followers.
RECOVER19 