<excerpt>
Situation
Information about robustness of scientific studies isn’t easily available.
Data Colada (possibly other groups) publish various kinds of robustness checks. e.g.,
- retraction
- replication
- (mine) all reqd fields consulted during review
- (others?)
Explore
- is there a formal definition of robustness?
- how are robustness and meta analyses connected?
- either way, what are other checks for robustness?
- what are there protocols of research sharing?
- how do you identify a paper is a meta analysis / robustness check
- can there be different types of links between papers?
- one paper refers two other papers. agrees with conclusions of one, disagrees with conclusions of the other. how are these differences recorded / logged / attested?
- which other
- low-hanging fruits:
- plugins for zotero, mendeley etc — flag robustness checks in search results, use check-wise filters, add to proofreading workflow
- plugins for crossref, doi, etc. — augment existing registries,
- protocols for sharing papers — similar to http headers
- scrape & process random papers
- datahub like platform to easily publish self-run checks
Initial answers
Retraction Watch has been acquired by Crossref. Not sure of similar replication registries.