Journal Metrics - Perspective from an Open Access Publisher

Presentation given at Leibniz Workshop on Publication Management

Transcript

1. Journal Metrics Perspective from an Open Access Publisher Martin Fenner Technical Lead Article-Level Metrics Public Library of Science Open Metrics http://pixabay.com/en/ruler-straight-edge-tool-geometry-145940/
2. Usage Stats Most immediate metric that directly reflects usage Only useful if data are collected in a standardized way – COUNTER is the standard and looks at HTTP status codes, double-click intervals, and excludes robots
3. http://open-access.net/fileadmin/OAT/OAT14/Tage-Koeln-Traue_keiner_Statistik_Recke_2014.pdf
4. 4 2854 34113 3247 2558 Scopus 
 Citations ≥ 10 HTML 
 Views ≥ 2000 Usage is different from scholarly citations Metrics collected August 8, 2012 42,772 PLOS ONE Papers
5. Citations http://dx.doi.org/10.1371/journal.pone.0063184 Citations have become a proxy for scholarly impact Many problems with unreflected use of citation metrics, in particular in the assessment of individual researchers
6. Citations are collected via reference lists Outcomes N All the data described in this article are available at http:// europepmc.org/ftp/oa/AccNoAnalysisData/. N The Whatizit ANA pipeline for ENA, UniProt and PDB accession numbers is integrated into the ePMC infrastructure and all the gathered accession numbers are available via the ePMC web site and web services (http://europepmc.org/ WebServices). N The extensions and improvements to the Whatizit ANA pipeline will be applied to the ePMC core program of named entity recognition and will be available via the web site and web services. N Tagged versions of the OA article set will be made available on an ongoing basis from the FTP site in the future. Acknowledgments We would like to thank the Rebholz research group at the EBI (2003-2012) for developing the Whatizit service, the EBI Literature Services Group for the development of many of the core data services used in this study, Andrew Caines for help producing the figures and Alex Bateman for critical reading of the manuscript. Author Contributions Conceived and designed the experiments: S ¸K JK JRM. Performed the experiments: S ¸K JK. Analyzed the data: S ¸K JK JRM. Wrote the paper: S ¸K JK JRM. References 1. Kahn P, Hazledine D (1988) NAR’s new requirement for data submission to the EMBL data library: information for authors. Nucleic Acids Res 16(10): I–IV. 2. Science as an Open Enterprise (2012) The Royal Society. Available: http:// royalsociety.org/policy/projects/science-public-enterprise/report/. Accessed 2013 Apr 8. 3. Rebholz-Schuhmann D, Arregui M, Gaudan S, Kirsch H, Yepes AJ (2007) Text processing through Web services: calling Whatizit. Bioinformatics 24(2):296– 298. 4. McEntyre JR, Ananiadou S, Andrews S, Black WJ, Boulderstone R, et al. (2011) UKPMC: a full text article resource for the life sciences. Nucleic Acids Res 39:d58–65. 5. Ne ´ve ´ol A, Wilbur WJ, Lu Z (2011) Extraction of data deposition statements from the literature: a method for automatically tracking research results. Bioinfor- matics 27(23):3306–3312. 6. Ne ´ve ´ol A, Wilbur WJ, Lu Z (2012) Improving links between literature and biological data with text mining: a case study with GEO, PDB and MEDLINE. Database (Oxford) 2012: bas026. 7. Fink JL, Kushch S, Williams PR, Bourne PE (2008) BioLit: integrating biological literature with databases. Nucleic Acids Res 36(Web Server Issue):W385–9. 8. Haeussler M, Gerner M, Bergman CM (2011) Annotating genes and genomes with DNA sequences extracted from biomedical articles. Bioinformatics 27(7):980–6. 9. Finn RD, Mistry J, Tate J, Coggill P, Heger A, et al. (2010) The Pfam protein families database. Nucleic Acids Res Database Issue 38:D211–222. 10. Parkinson H, Sarkans U, Shojatalab M, Abeygunawardena N, Contrino S, et al. (2005) ArrayExpress – a public repository for microarray gene expression data at the EBI. Nucleic Acids Res (2005) 33 (Suppl 1): D553–D555. 11. Mulder NJ, Apweiler R, Attwood TK, Bairoch A, Bateman A, et al. (2002) InterPro: an integrated documentation resource for protein families, domains and functional sites. Brief Bioinform (3):225–35. http://dx.doi.org/10.1371/journal.pone.0063184
7. Reference lists have to be collected in a central resource and in a standard format
8. http://www.crossref.org/01company/02history.html CrossRef's specific mandate is to be the citation linking backbone for all scholarly information in electronic form
9. Limitations CrossRef citation linking built around references that have DOIs from CrossRef members – usually scholarly articles CrossRef Cited-By service only available to CrossRef members for their own articles CrossRef is a non-profit organization with publishers as members – no academic institutions, funders, other stakeholders
10. Alternative citation indexes for non-publisher users
11. Reference lists increasingly contain non-article references http://dx.doi.org/10.1371/journal.pone.0115253 Loss of Current and Past Context In this experiment, we aim at providing an insight into the loss of current and past Fig. 3. STM articles and URI references per publication year - PMC corpus. doi:10.1371/journal.pone.0115253.g003 Scholarly Context Not Found
12. DataCite provides DOIs for academic institutions and data centers
13. DataCite DOIs are not just for datasets http://datacite.labs.orcid-eu.org/help/status
14. DataCite citation linking works differently, and is separate from CrossRef A DOI is not a DOI http://datacite.labs.orcid-eu.org/help/status
15. CrossRef and DataCite announce initiative Nov 2014 • Provide comprehensive support for interlinking between articles and data. • Develop open APIs and open source tools to surface citations and other relationships between publications and data sets. • Integrate into their services other existing scholarly communications initiatives such as ORCID and FundRef. • Develop systems, workflows and best practices for using DOIs to reference large, highly granular and dynamic data. https://www.datacite.org/CrossRefDataCiteinitiative
16. There is more than 
 usage stats and citations RESEARCH ARTICLE VIEWED SAVED DISCUSSED RECOMMENDED CITED PLOS HTML PLOS PDF PLOS XML PMC HTML PMC PDF CiteULike Mendely NatureBlogs ScienceSeeker ResearchBlogging PLOS Comments Wikipedia Twitter Facebook F1000 Prime CrossRef PMC Web of Science Scopus Increasing Engagement http://dx.doi.org/10.3789/isqv25no2.2013.04
17. Days since publication August 27, 2014 PLOS collects metrics from 22 data sources http://alm.plos.org/articles/info:doi/10.1371/journal.pone.0105948
18. New metrics not ready for impact assessment Any metric we use should have good reliability (consistency) and validity. More work is needed in these areas for novel assessment metrics such as Mendeley or Twitter. http://en.wikipedia.org/wiki/Validity_(statistics)#mediaviewer/File:Reliability_and_validity.svg
19. Work on best practices and standards has started Alternative Metrics Initiative Phase 1 White Paper June 6, 2014 http://www.niso.org/topics/tl/altmetrics_initiative/ Phase II of the project starts in early 2015
20. Altmetrics data can be obtained from commercial service providers
21. https://github.com/articlemetrics/alm-report Open source software to collect and analyze the metrics data https://github.com/articlemetrics/lagotto
22. CrossRef DOI Event Tracker (DET) Pilot CrossRef Labs has started a pilot project to collect events around all CrossRef DOIs issued since January 2011. A DOI Event Tracker (DET) CrossRef working group was formed in May 2014, initiated by members of the Open Access Scholarly Publishers Association (OASPA). The service is available at http://det.labs.crossref.org, and is using the Lagotto open source software.
23. This presentation is made available under a CC-BY 4.0 license. http://creativecommons.org/licenses/by/4.0/