Reproducible Research: ideas & solutions for actually doing it

Labchat preentation given to EvoHull group on techniques for achieving reproducible research

Transcript

1. reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreprodu reproducible research ideas and solutions for actually doing it Dave Lunt June 2016 dave.lunt@gmail.com Labchat is reproducible, all resources are at: https://osf.io/cqb6x/
2. http://xkcd.com/ reproducible research is science
5. Reproducibility makes your life much easier ‘future you’ will reproduce your work reproducibility gives you new experimental ‘powers’ carrot not stick
6. reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreprodu small wins remember past successes tolerate minor inconvenience iterate know your issues don’t reinvent the wheel do something now implementing reproducibility
7. reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreprodu tolerate minor inconvenience remember past successes small wins don’t reinvent the wheel do something now know your issues iterate What do you want from reproducibility?
8. reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreprodu tolerate minor inconvenience remember past successes small wins don’t reinvent the wheel do something now know your issues iterate What do you want from reproducibility? OK, now be more ambitious...
9. ACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATGATCGAC TACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATACGATCGA TCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAATCGAGCG ACACGAACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATG ATCGACTACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATAC GATCGATCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAAT CGAGCGACACGAACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGA TCGATGATCGACTACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCG ATATACGATCGATCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGA TCGAATCGAGCGACACGA ACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATGATCGAC TACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATACGATCGA TCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAATCGAGCG ACACGA ACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATGATCGAC TACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATACGATCGA TCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAATCGAGCG ACACGA ACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATGATCGAC TACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATACGATCGA TCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAATCGAGCG ACACGA ACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATGATCGAC TACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATACGATCGA TCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAATCGAGCG ACACGA ACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATGATCGAC TACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATACGATCGA TCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAATCGAGCG ACACGA ACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATGATCGAC TACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATACGATCGA TCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAATCGAGCG collaboration archiving credibility backup provenance ease publication
10. reproducibility environments what you should use
11. reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreprodu git osf docker jupyter figshare galaxy Protocols.io
12. reproducibility environments Things that work for starting out reducing reproducibility friction
13. reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreprodu git osf docker jupyter figshare galaxy Protocols.io
14. https://osf.io
16. http://osf.io OSF is imperfect but very workable small wins remember past successes tolerate minor inconvenience iterate know your issues don’t reinvent the wheel do something now
17. http://osf.io OSF is imperfect but very workable small wins remember past successes tolerate minor inconvenience iterate know your issues don’t reinvent the wheel do something now Its pretty good actually, easy to use You could start today, achieve small wins
18. https://osf.io/cqb6x/ live demo
19. http://protocols.io Protocols are open cc-by Protocols have doi
23. reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep roducibilityreproducibilityreproducibilityreproducibilityreproducibility reproducibilityreproducibilityreproducibilityreproducibilityreproducib ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod ucibilityreproducibilityreproducibilityreproducibilityreprodu small wins remember past successes tolerate minor inconvenience iterate know your issues don’t reinvent the wheel do something now know your issues, start small, and do something now [the end]