A look inside the black box of scientific publishing

Publishing the results of one’s research is an integral part of the scientific process, yet scholarly journals are often seen as black boxes by researchers. What happens to a paper after it is submitted? Who is deciding on its fate? What is the role of the journal editor and the editorial office? How does the peer-review process work, and are its core principles still relevant in today’s changing publishing landscape? In this talk I will discuss these questions in an attempt to de-mystify the peer review process from an editor's perspective, and cover the whats, the hows and the whys of peer review.

Transcript

1. Wiley A look inside the black box of scientific publishing Dr Matteo Cavalleri, Publisher, Materials Science & Physics @physicsteo linkedin.com/teowaits
2. Who I am M.S. in Chemistry (1999) PhD in Chemical Physics (2005) PostDoc 2005-2008
3. Publishing is a career for PhDs Peer-Review Editor (2008-2010), Berlin Associate Editor (2010-2012), NYC Editor-in-Chief (2012-2020), NYC Executive Editor (2017-2020) & Publisher (Present)
4. Where I work HOBOKEN, NJ • Founded in 1807 in NYC • Headquarter in Hoboken, NJ • Publicly listed in NYSE • ~5000 staff worldwide • ~1600 journals • ~9000 books …in partnership with 1085 organizations (865 scholarly societies, + institutes, universities, goverments,…)
5. Where I work HOBOKEN, NJ • Founded in 1807 in NYC • Headquarter in Hoboken, NJ • Publicly listed in NYSE • ~5000 staff worldwide • ~1600 journals • ~9000 books …in partnership with 1085 organizations (865 scholarly societies, + institutes, universities, goverments,…) View from the office (not my office) #WorkFromHome
6. What I do all day Publisher (Present) • International in-house editorial team of 7, based in East Coast of US • Lead US in-house editorial team (co-founded in 2010) • Publisher of the Materials Science & Physics team • Portfolio includes OA & subscription titles !"#$%&
7. Why publish? •Fame •Recognition by peers •Fortune •Promotions •Grant applications •Establish precedence •Responsibility •Taxpayer-funded research • Making your research public • “If your research does not generate papers, it might just as well not have been done.” –George Whitesides • Papers provide the shoulders that others can stand on The “Publish or Perish” culture responsible for an environment where (some) scientific journals are de facto gatekeepers of science
8. Why journals? Dissemination Spreading the word through publishing platforms But also indexing and generally organizing knowledge Registration Precedence of discovery is established based on article submission date to a journal Archival Safeguarding and preserving knowledge Publishers play an important role preserving the scientific record Certification Peer-review is still the gold standard for certifying articles This is not the same as quality-control! Peer-review management, Curation, Infrastructure, Ethics, & much, much more. Here’s a list of 96 things publishers do: https://bit.ly/2UW3rKX
9. How do journals work?
10. How do journals work? By Nick Kim (www.nearingzero.net); used with permission
11. What is the peer-review process? “Peer review is the critical assessment of manuscripts submitted to journals by experts who are not part of the editorial staff”-International Committee of Medical Journals Editors WHAT IT CANNOT DO (*) WHAT IT SHOULD DO -Filter out bad/uninteresting work -Make as sure as possible the work is reported correctly -Make sure results are interpreted correctly, and convincingly -Improve the quality of publication -Detect fabrication -Prevent duplicate publication -Pick the most interesting papers -Ensure quality -Ensure the article is right for the journal (*) AUTOMATICALLY
12. How did we get here? “We (Mr. Rosen and I) had sent you our manuscript for publication and had not authorised you to show it to specialists before it is printed. I see no reason to address the – in any case erroneous – comments of your anonymous expert.” 1665: Publication of Journal des Sçavans and Philosophical Transactions of the Royal Society. 1930s-1940s: Peer-review emerges, not very popular at first 1960s: Peer-review becomes the institutional standard (Nature, 1967) Today: Enjoy overwhelming support despite issues*, alternatives emerge * Slow and costly, systemic biases, still misses fraudulent papers
13. Peer-Review Types -SINGLE BLIND: Most common -DOUBLE BLIND: Medical journals -OPEN: Atmospheric Chemistry and Physics -SIGNED: Non-anonymous referees, BMJ Open -TECHNICAL PEER-REVIEW ONLY: PLoS One, Scientific Reports, PeerJ,… -MIX OF THE ABOVE: Independent/Interactive, “Frontiers In”, EMBO… -NONE: Evaluation by community post-publication, arXiv, F1000,… -INDIPENDENT FROM JOURNAL: Rubiq, Peerage of Science -PORTABLE: Within the same publishing house, portfolio of journals -TRANSPARENT: Report published with accepted article
15. The editorial office EXTERNAL EDITORS IN-HOUSE EDITORS …all ACS, T&F, OUP, most Springer-Nature, Elsevier, Wiley journals… …+ some APS, RSC, IOP titles, Cell Press, Science, PLOS, The Lancet…
16. The editorial office EXTERNAL EDITORS IN-HOUSE EDITORS Have PhDs, often PostDoc experience Work full time on journal – can dedicate more time and resources on new developments General view Have own research group Expert in specific field BOTH: peer-review, decision making, dealing with appeals, commissioning, conference participation and lab visits, writing news stories, contributing to “input” marketing …
17. What editors look for? MOST JOURNALS -Novelty -Importance (in specific field / in related disciplines) -Interest ALL JOURNALS -Scope -Format (Communication, full paper, review…) -Understandability -Compliance to guidelines, ethical behavior Editors are not always qualified to evaluate the technical merits of manuscripts. This is the job of the referees.
18. How are referees chosen? -Editors’ knowledge & experience -From related papers: - cited manuscripts - literature search -Additional research: - conference/lab visits - web search (good ‘ol Google) -Reviewer database: - keywords, interest, history…
19. Referee suggestions are welcome -Not just the big names, please -No collaborators, previous advisors, grant co-applicants, … -Tell us about circumstances that may prevent impartial review: - close competitors, who may “scoop” you - other conflicts …within reason… Follow the 3W rule when suggesting reviewers: Tell the editor WHO, WHERE, WHY
20. Accept, reject, or revise? -REJECTION - Without external referee reports (Editor) - Based on reports -REVISION - Reconsideration or resubmission possible after major revisions -ACCEPTANCE - Without changes (rare) - With minor changes The decision is the Editor’s job…the reviewer ‘s recommendation is not a vote -- it’s advice!
21. Revision -REVISION • Carefully consider referee comments • Not all changes have to be made… • …but need convincing arguments for changes not made • Prepare revision • Revise manuscript • Highlight changes in manuscript • Point-by-point response to all referee criticisms • Changes made • Why changes not made • Response may go back to referees! • Need to convince editor and referees The peer-review process is not a private conversation between authors and referees. Try to work your answers to the reviewers in the revised manuscript!
22. Rejection – not the end of the world! -REJECTION • Most scientists have been rejected– do not take it personally • Try to understand why the paper was rejected • Note that you have received the benefit of the Editors and reviewers’ time: take their advice seriously! • Re-evaluate your work • If you resubmit, begin as if you are going to write a new article • Consider offers to transfer for your manuscript to another related journal
23. Reject and Transfer -REJECTION -Sharing referee reports and communication among editors allow research to find the right audience -Fast publication, often without further peer- review (if rejected on reports) -40% of submissions/year (~1.3M manuscripts!) are rejected after peer-review (Rubriq, 2013). Often by the same reviewers. Impact Factor 12.256 Impact Factor 3.075
24. Appeal USUALLY, NO -Risk of long time of publication -Good papers are found and cited -Editors & referees know journal well OCCASIONALLY, YES -Importance/novelty missed by editor/referees -Factual error in referee reports that lead to rejection -Need more clarification of decision Be calm, argumentative and give scientific justification for reassessment It’s false that Editors hate appeals. So should you appeal a rejection? Dear Dr. *****: Regarding your decision on my submission **-*****, I am afraid you and the referees are crazy. Please reconsider your decision. Best wishes -REJECTION
25. We are pleased to inform you…
26. Congratulations! -ACCEPTANCE • Production data checklist • Text format • Figure preparation • Proofs! • Return quickly! • Check CAREFULLY copyediting changes/queries • Ask a friend to check as well • NOW pop the prosecco bottle! Author Correction Early View Online Publication Issue Build and checking Issue Publishing and Distribution Typesetting Submission Peer review Copy- editing In the editorial office:
27. Welcome to the future…
28. Publishing is changing Science has changed a lot since the 17th century, but research dissemination not too much • Print is on the way out, how about the PDF? • Open Science & Open Access • Data sets and other forms of scientific output (codes!) as citable units • Referee’s reports are part of the discourse • Article metrics >> Journal metrics • Articles are discovered outside of issues • Demographic shift of authorship • Correct identification of authors is paramount! https://onlinelibrary.wiley.com/page/journal/1097461x/homepage/interactivearticle_vi.html
29. Search Engine Optimization (SEO) for your article • Search-Engine friendly Title/Abstract • Use keywords throughout the article • Be consistent with authors names • In-bound links rule Google. Link your article across social media, networking and institutional sites • Network, highlight/elevate your colleagues, they will do the same for you! • Share data, code. Open science leads to greater collaboration, increased confidence in results and goodwill between researchers • Most journals welcome preprints! HELP PEOPLE FIND YOU https://authorservices.wiley.com/author-resources/Journal-Authors/Prepare/writing-for-seo.html
30. Open Access A growing, global, movement 800+ OA policies in place globally Grounded in shared Declarations Supported by policy changes 0 20 40 60 80 100 120 2011 2012 2013 2014 2015 2016 2017 2018 2019 Substantiated by a growing OA portfolio
31. Be identified, be credited Persistent digital identifier that distinguishes researchers from each other. http://orcid.org High-level taxonomy that can be used to represent the roles typically played by contributors to scientific scholarly output. https://casrai.org/credit.html
32. Tl;dr THIS IS NOT HOW IT WORKS! Dr Matteo Cavalleri @physicsteo linkedin.com/teowaits mcavalleri@wiley.com