Fear Factor: Use, misuse, and abuse of the impact factor

Internal presentation (Lunch & Learn, 2019) on the Journal Impact Factor. What it is, and what is not.

Transcript

1. Fear Factor: Use, misuse, and abuse of the impact factor MATTEO CAVALLERI Editor-in-Chief International Journal of Quantum Chemistry Publisher John Wiley & sons, inc. @physicsteo
2. Show of hands • Do you know what the impact factor is, and how it is calculated? • Do you think higher impact factor = better quality journa?l • Do you heavily rely on the impact factor to advertise your journals to editors, authors, reviewers? • Do you know what the h-index is?
3. Impact Factor Definitions (EUGENE) GARFIELD, SCIENCE 178:471-479 (1972) JCR DEFINITION 1972 DEFINITION “Dividing the number of times a journal has been cited by the number of articles it has published during some specific period of time. The journal impact factor reflect an average citation rate per article” -“All citations to the journal to items published in the previous two years divided by the total number of scholarly items published in the journal in the previous two years” 2018 IF TOTAL CITATIONS in 2018 TO ITEMS PUBLISHED in 2016 & 2017 # of SCHOLARLY ITEMS (ARTICLES, REVIEWS, PROCEEDINGS) PUBLISHED in 2016 & 2017
4. Impact Factor Definitions (EUGENE) GARFIELD, SCIENCE 178:471-479 (1972) JCR DEFINITION 1972 DEFINITION “Dividing the number of times a journal has been cited by the number of articles it has published during some specific period of time. The journal impact factor reflect an average citation rate per article” -“All citations to the journal to items published in the previous two years divided by the total number of scholarly items published in the journal in the previous two years” 2.263 722 319
5. Impact Factor Meaning A MEASURE OF AVERAGE CITATION RATE OF A JOURNAL WHAT IT IS 2018 IF TOTAL CITATIONS in 2018 TO ITEMS PUBLISHED in 2016 & 2017 # of SCHOLARLY ITEMS (ARTICLES, REVIEWS, PROCEEDINGS) PUBLISHED in 2016 & 2017 • Intuitive • Universally accepted / imitated • Not uncontroversial • Measure of journal prestige • Field specific • Skewed towards higher cited articles • Open to manipulation J. Am. Soc. Inf. Sci., 60: 27-34. doi:10.1002/asi.20936 arXiv:1906.02660
6. Impact Factor Meaning A MEASURE OF AVERAGE CITATION RATE OF A JOURNAL WHAT IT IS • Intuitive • Universally accepted / imitated • Not uncontroversial • Field specific • Measure of journal prestige • Skewed towards higher cited articles • Open to manipulation WHAT IT ISN’T • Transparent (underlying databased not publicly available) • Measure of quality of individual article / researcher • Measure of prestige of an institution • Predictor of likely citations @DuttonChemistry, Jason Dutton, Assoc. Prof. Latrobe University
7. Impact Factor Misuse ESPECIALLY AS SOLE METRICS • Use in hiring/promotion/tenure decisions • Use in grant evaluations • Employed in University Rankings
8. Impact Factor Abuse WITH GREAT POWER COMES GREAT RESPONSIBILITY • Impact Factor as tool for evaluating researchers’ productivity has altered the research landscape • Association with fraud BMJ 2007; 334 doi: 10.1136/bmj.39142.454086.AD PNAS, doi:10.1073/pnas.1212247109
9. Impact Factor Abuse WITH GREAT POWER COMES GREAT RESPONSIBILITY • Impact Factor as tool for evaluating researchers’ productivity has altered the research landscape • Association with fraud BMJ 2007; 334 doi: 10.1136/bmj.39142.454086.AD PNAS, doi:10.1073/pnas.1212247109
10. H-Index JE HIRSCH, PNAS, 102(46):16569 -16572 (2005) Metric of the scientific output of an individual, “…incorporating both quantity and visibility of publication” • H-index of 10 means having published 10 articles having at least 10 citations each • Field-dependent • Never decreases • No time-window • Insensitive to citation extremes • Insensitive to author’s contribution to article • Does not correct for self-citations
11. H-Index WITH GREAT POWER COMES GREAT RESPONSIBILITY • Simplistic, but measures the broad impact of one’s work • Should not be used as (sole) criterion to inform decision making in science PHYSICIST B PHYSICIST A H-index @ PhD: 5 10
12. TL;DR DON’T WORRY SO MUCH ABOUT THE IMPACT FACTOR • The absolute value of the impact factor is meaningless • It’s not about people, or articles, it’s about journals • No single metrics should be applied alone to evaluation of journals, research • We have a role to play in avoiding misuses and abuses • Matteo measurably better physicist than Einstein