A peek inside the black box of scientific publishing

Publishing the results of one’s research is an integral part of the scientific process, and of scientists’ careers, yet scholarly journals are often seen as black boxes by researchers.

With this talk I aim to reduce the opacity of the inner working of scientific journals and to de-mystify the peer review process from an editor's perspective. I’ll also share my (very happy) experience going from a computational chemistry lab to my current career on the “other side” of scientific publishing, and explore roles for STEM Ph.D.s in the publishing industry.

Transcript

1. A peek inside the black box of scientific publishing Dr Matteo Cavalleri, Publisher, Materials Science & Physics e-Girona, April 27, 2021 Wiley @physicsteo linkedin.com/teowaits mcavalleri@wiley.com
2. Who I am M.S. in Chemistry (1999) PhD in Chemical Physics (2005) PostDoc 2005-2008
3. My career ”on the other side” Peer-Review Editor (2008-2010), Berlin Associate Editor (2010-2012), NYC Editor-in-Chief (2012-2020), NYC Rich media objects (data, code, notebooks, and interactive visualizations) integrated in the Version of Record The “Article of the Future” (IJQC, 2020)
4. “The Article of the Future” Data are connected to the figure that describe them. They are discoverable, indexable, citable, and usable The code is in the figure, in an executable time capsule Peer review reports, author rebuttals and editor notes get a DOI and can be cited Experience the future: http://bit.ly/IJQC_AotF
5. My career “on the other side” (2020-) New IJQC Editor-in-Chief”: Gian Asara Publisher (Present) PhD @ UB w/ Prof. J.M. Ricart and Prof. F. Illas • International in-house editorial team of 7, based in East Coast of US • Lead US in-house editorial team (co-founded in 2010) • Publisher of the Materials Science & Physics team • Portfolio includes OA & subscription titles !"#$%&
6. Where I work 1807 mid 1800s Today Charles Wiley opened a print shop in New York City, publishing literary fiction and non-fiction. John Wiley & Sons began focusing on science, technical, and engineering publishing. Seven generations later, Wiley is one of the oldest independent publishing companies. 1700+ journals, 400+ Open Access journals, 7.5M articles 500+ societies, 2M members #WorkFromHome
7. How do journals work?
8. How do journals work? By Nick Kim (www.nearingzero.net); used with permission
9. How do journals work?
10. How do journals REALLY work? Dissemination Spreading the word through publishing platforms But also indexing and generally organizing knowledge Registration Precedence of discovery is established based on article submission date to a journal Archival Safeguarding and preserving knowledge Publishers play an important role preserving the scientific record Certification Peer-review is still the gold standard for certifying articles This is not the same as quality-control! Peer-review management, Curation, Infrastructure, Ethics, & much, much more. Here’s a list of 96 things publishers do: https://bit.ly/2UW3rKX The “Publish or Perish” culture responsible for an environment where (some) scientific journals are de facto gatekeepers of science
11. Author submits article Rejected Article assessed by editor Sent to reviewers Author submits revised paper Revision required Further review needed? Reviews assessed by editor Rejected Accepted Publication Production How do journals REALLY work?
12. How do journal REALLY work? “Peer review is the critical assessment of manuscripts submitted to journals by experts who are not part of the editorial staff”-International Committee of Medical Journals Editors
13. How did we get here? “We (Mr. Rosen and I) had sent you our manuscript for publication and had not authorised you to show it to specialists before it is printed. I see no reason to address the – in any case erroneous – comments of your anonymous expert.” 1665: Publication of Journal des Sçavans and Philosophical Transactions of the Royal Society. 1930s-1940s: Peer-review emerges, not very popular at first 1960s: Peer-review becomes the institutional standard (Nature, 1967) Today: Enjoy overwhelming support despite issues*, alternatives emerge * Slow and costly, systemic biases, still misses fraudulent papers
14. Peer-Review Types -SINGLE BLIND: Most common -DOUBLE BLIND: Authors undisclosed to reviewers -OPEN: Public, community peer-review -SIGNED: Non-anonymous referees, BMJ Open -TECHNICAL PEER-REVIEW ONLY: PLoS One, Scientific Reports, PeerJ,… -MIX OF THE ABOVE: IJQC experiment, “Frontiers In”, EMBO… -NONE: Evaluation by community post-publication, arXiv, F1000,… -INDIPENDENT FROM JOURNAL: Rubiq, Peerage of Science -PORTABLE: Within the same publishing house, portfolio of journals -TRANSPARENT: Report published with accepted article Credit: Andrew Bissette, @andrewbissette -TRIPLE BLIND: Authors undisclosed to reviewers and editors
15. The editorial office EXTERNAL EDITORS IN-HOUSE EDITORS …all ACS, T&F, OUP, most Springer-Nature, Elsevier, Wiley journals… …+ some APS, RSC, IOP titles, Cell Press, Science, PLOS, The Lancet…
16. The editorial office EXTERNAL EDITORS IN-HOUSE EDITORS Have PhDs, often PostDoc experience Work full time on journal – can dedicate more time and resources on new developments General view Have own research group Expert in specific field BOTH: peer-review, decision making, dealing with appeals, commissioning, conference participation and lab visits, writing news stories, contributing to “input” marketing …
17. What editors look for? MOST JOURNALS -Novelty -Importance (in specific field / in related disciplines) -Interest ALL JOURNALS -Scope -Format (Communication, full paper, review…) -Understandability -Compliance to guidelines, ethical behavior Editors are not always qualified to evaluate the technical merits of manuscripts. This is the job of the referees.
18. How are referees chosen? -Editors’ knowledge & experience -From related papers: - cited manuscripts - literature search -Additional research: - conference/lab visits - web search (good ‘ol Google) -Reviewer database: - keywords, interest, history…
19. Referee suggestions are welcome -Not just the big names, please -No collaborators, previous advisors, grant co-applicants, … -Tell us about circumstances that may prevent impartial review: - close competitors, who may “scoop” you - other conflicts …within reason… Follow the 3W rule when suggesting reviewers: Tell the editor WHO, WHERE, WHY
20. Accept, reject, or revise? -REJECTION - Without external referee reports (Editor) - Based on reports -REVISION - Reconsideration or resubmission possible after major revisions -ACCEPTANCE - Without changes (rare) - With minor changes The decision is the Editor’s job…the reviewer ‘s recommendation is not a vote -- it’s advice!
21. Revision -REVISION • Carefully consider referee comments • Not all changes have to be made… • …but need convincing arguments for changes not made • Prepare revision • Revise manuscript • Highlight changes in manuscript • Point-by-point response to all referee criticisms • Changes made • Why changes not made • Response may go back to referees! • Need to convince editor and referees The peer-review process is not a private conversation between authors and referees. Try to work your answers to the reviewers in the revised manuscript!
22. Rejection – not the end of the world! -REJECTION • Most scientists have been rejected– do not take it personally • Try to understand why the paper was rejected • Note that you have received the benefit of the Editors and reviewers’ time: take their advice seriously! • Re-evaluate your work • If you resubmit, begin as if you are going to write a new article • Consider offers to transfer for your manuscript to another related journal
23. Reject and Transfer -REJECTION -Sharing referee reports and communication among editors allow research to find the right audience -Fast publication, often without further peer- review (if rejected on reports) -40% of submissions/year (~1.3M manuscripts!) are rejected after peer-review (Rubriq, 2013). Often by the same reviewers.
24. Appeal USUALLY, NO -Risk of long time of publication -Good papers are found and cited -Editors & referees know journal well OCCASIONALLY, YES -Importance/novelty missed by editor/referees -Factual error in referee reports that lead to rejection -Need more clarification of decision Be calm, argumentative and give scientific justification for reassessment It’s false that Editors hate appeals. So should you appeal a rejection? Dear Dr. *****: Regarding your decision on my submission **-*****, I am afraid you and the referees are crazy. Please reconsider your decision. Best wishes -REJECTION
25. General structure of a scientific article 1. Have something to say 2. Say it 3. Stop as soon as you have said it. Billings, J., An address to our medical literature. British Medical Journal 1881, 262-268
26. Figures & Tables = Your paper’s storyboard Figure 1 Figure 2 Figure 3 ….. Figure 7 Figure 8 Star Wars, Episode V; The Empire Strikes Back
27. The A,B,C of good scientific writing • Avoid vague language and be precise/specific • Say EXACTLY what you mean and avoid over/under statements • Make the discussion concise but informative. Focus on the important and unexpected results. Not on small details. • Use as few words as possible while retaining meaning without sacrificing scientific details Use simple words and avoid jargon Use verb tense consistently throughout the paper Where possible, use verbs instead of noun forms
28. Search Engine Optimization (SEO) for your article • Search-Engine friendly Title/Abstract • Use keywords throughout the article • Be consistent with authors names • In-bound links rule Google. Link your article across social media, networking and institutional sites • Network, highlight/elevate your colleagues, they will do the same for you! • Share data, code. Open science leads to greater collaboration, increased confidence in results and goodwill between researchers • Most journals welcome preprints! HELP PEOPLE FIND YOU https://authorservices.wiley.com/author-resources/Journal-Authors/Prepare/writing-for-seo.html
29. One editor’s origin story
30. Journal Editor: A day in the life* OUTREACH DAY-TO-DAY MANAGEMENT OF A JOURNAL • Read manuscript submitted to the journal • Identify and assign manuscript to reviewers (or sub-Editors) • Make editorial decisions • Solicit manuscripts and special issues • Deal and resolve (hopefully!) ethical issues regarding submissions/published papers • Collaborate with editorial, production, marketing team • Work closely with authors, reviewers, other editors • Beta testing new publishing technologies • Help authors to disseminate their work further • Attend international conferences • Workshops, lab/institution visits *Work days are pretty reasonable, actually
31. In-house editors: Life as a pro …AND WHAT WE ARE NOT… • Active in research (not even part-time!) • Lone wolves & hermits WHAT WE ARE… • Full-time employees of a publisher • Former researchers (PhD, often PostDoc experience) • Working in small editorial team (1~20 ppl), supported by editorial board • Work (mostly, especially initially) from the publisher’s offices* • Holding different titles (EiC, associate,..) depending on experience & responsibility • Responsible for peer-review process and editorial decisions • Responsible for content commissioning • The face of the publisher in the community, voice of researchers in the publisher *pre-COVID, at least
32. Publisher*: A day in the life LAUNCH/ACQUISITION OF NEW PRODUCTS • New journals • Journal relaunches • Awards, conferences, workshops • Webinars, online courses STRATEGIC MANAGEMENT AND DEVELOPMENT OF JOURNALS • Hire, train, and manage editors and editorial assistants • Roll out new publishing initiatives and processes • Planning, budgeting, and reporting (Editors don’t deal with finance!) • Develop common strategies and goals for portfolio/groups of journals • Bring the perspective of the researchers to the company • Collaborate closely with editors, marketers, other colleagues *”WILEY Publisher”. Titles are not standard in the industry
33. Other roles: Not just peer-review MANAGING EDITOR/JOURNAL PUBLISHING EDITOR • Scientific background (sometimes, but not always) • Support academic EiC, liaison with production and marketing team • Can be involved in making desk rejections DEVELOPMENTAL/ACQUISITION EDITOR • Scientific backgrounds (often) • Commission articles and special issues • Responsible for acquisition/launch/improvement of scholarly products • Common role in book publishing COPY EDITOR • Scientific background (often at MS/BS level, not PhD) • Responsible for proof-reading manuscripts • Mostly present in ”apex” titles, major brands
34. Other roles: Not just peer-review MANAGING EDITOR/JOURNAL PUBLISHING EDITOR • Scientific background (sometimes, but not always) • Support academic EiC, liaison with production and marketing team • Can be involved in making desk rejections DEVELOPMENTAL/ACQUISITION EDITOR • Scientific backgrounds (often) • Commission articles and special issues • Responsible for acquisition/launch/improvement of scholarly products • Common role in book publishing COPY EDITOR • Scientific background (often at MS/BS level, not PhD) • Responsible for proof-reading manuscripts • Mostly present in ”apex” titles, major brands
35. What’s hot & what’s not …AND WHAT I WOULD DO WITHOUT • Journal/process development can be slow and frustrating • Angry authors are difficult to deal with • Fraud/Ethical violations are not uncommon and very exasperating! • Sometimes I miss coding, hacking hardware (being a “lab-rat”) • Career progression after Editor-in-Chief/JPE not easy WHAT I LOVE… • It’s a career at the “center of science” • Entrusted the knowledge of entire disciplines • Bird’s-eye view over science, see best results 1st! • Contact with the scientific community • Add & participate to the scientific debate and progress • Plenty of (international) travel* • Real possibility of professional growth *pre-COVID, at least
36. What is a good editor made of? … BUT YOU WON’T LOVE IT IF YOU … • love being in the lab and doing research • enjoy being the world expert in a specific subject • don’t like changing topics several times a day • hated writing your thesis IT MAY BE THE JOB FOR YOU IF YOU … • are passionate for science communication • recognize the importance of publishing in the scientific process • are curious about a broad range of topics & disciplines • know the art of diplomacy and have people skills • have analytical and decision-making skills • are creative, with an eye for detail (and the “next big thing”) ENGLISH IS THE LANGUAGE OF SCIENCE • Publishing not restricted to native speakers anymore • BUT, you need to be fluent in communicating science with it
37. Dr Matteo Cavalleri @physicsteo linkedin.com/teowaits mcavalleri@wiley.com
38. Wiley is wonderful. Really, but other places are available…. and more… In-house editors wanted: Other roles: