Research on data journalism: What is there to investigate? Insights from a structured literature review

Presented: April 21, 2016; University of Helsinki
at the Northern Data Journalism Conference (NODA16) Academic Pre-Conference

This presentation aims at exploring the existing research literature on data journalism. Over the past years this emerging journalistic practice has been established and has also attracted significant attention from journalism scholars. It was time to take a closer look at the existing research literature in order to find out more about how this literature has been developing. Where are the research gaps and what does the future of data journalism research hold? These questions were tackled by carefully selecting a corpus of scholarly literature with empirical foundation in data journalism. This corpus was analyzed with a mixed method approach using qualitative and quantitative techniques. In this way the development of the literature over time could be illustrated and the most influential publications could be identified. Often-used theoretical frameworks and the applied research designs hinted at certain tendencies and gaps in the research literature on data journalism, for example, the dominance of qualitative research design over quantitative ones. Also, a shortcoming of cross-national investigations and ethnographic studies became visible.

Transcript

1. MEDIA & DESIGN Research on data journalism: What is there to investigate? Insights from a structured literature review Julian Ausserhofer1,2,3, Robert Gutounig1, Michael Oppermann2, Sarah Matiasek1,2 & Eva Goldgruber1 1: FH Joanneum University of Applied Sciences, Graz
 2: University of Vienna
 3: Humboldt Institute for Internet and Society, Berlin NODA16 Academic Pre-Conference
 #NODA16
 21.04.2016, University of Helsinki
2. MEDIA & DESIGN *Tool evaluation partners * * * Supported by:
3. MEDIA & DESIGN Research Interest
4. MEDIA & DESIGN @julauss Research literature on data journalism 2013: "internalist tendencies at [... the] early stage of academic research" (Anderson, 2013, p. 1007) ↓
 
 2015: "an explosion in data journalism- oriented scholarship" (Fink & Anderson, 2015, p. 476)*
 
 "rapidly growing body" of scientific studies 
 (Lewis, 2015, p. 322)*
 
 
 
 *cited via Loosen, Reimer & Schmidt (2015, p. 2) 4 #NODA16
5. MEDIA & DESIGN How is the research literature developing? What are the research gaps? Research questions @julauss 5 #NODA16
6. MEDIA & DESIGN Method
7. MEDIA & DESIGN Structured literature review @julauss 7 #NODA16
8. MEDIA & DESIGN "to develop insights, critical reflections, future research paths and research questions" 
 (Massaro, Dumay & Guthrie, forthcoming) It adopts "a replicable, scientific and transparent process [...] that aims to minimize bias [...]"
 (Tranfield, Denyer & Smart, 2003) Why a structured literature review? @julauss 8 #NODA16
9. MEDIA & DESIGN Undertaking a systematic literature review 9 Adapted from Massaro et al. (forthcoming) Writing a literature review protocol Developing insights and critique through analyzing the dataset Developing future research paths and questions Determining the type of studies and carrying out a comprehensiv e literature search Coding data Defining the questions that the literature review should answer @julauss #NODA16
10. MEDIA & DESIGN 10 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensiv e literature search • Empirical research on DDJ • Social science focus, but open to other disciplines • Published after 1995 Included Journal articles Book sections Conference papers Reports (from industry and research projects) PhD theses Not included Bachelor's and Master's theses Press reports Blog posts @julauss #NODA16
11. MEDIA & DESIGN 11 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensiv e literature search • Preliminary search with “data-driven journalism” • Extracting related terms from the keyword section of research papers @julauss #NODA16
12. MEDIA & DESIGN 12 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensiv e literature search Search terms algorithmic journalism computational journalism computer-assisted reporting data journalism data-driven journalism data-driven reporting database journalism datajournalism datenjournalismus quantitative journalism No search terms accountability journalism crowdsourced journalism dataviz datavis ddj drone journalism investigative journalism online journalism open journalism @julauss #NODA16
13. MEDIA & DESIGN 13 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensiv e literature search Scientific Databases ACM Digital Sowiport EBSCO Springer IEEE SpringerLink JSTOR Taylor & Francis Online ProQuest Web of Science Science Direct Wiley Scopus Google Scholar Sociological Abstracts @julauss #NODA16
14. MEDIA & DESIGN 14 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensiv e literature search 772 search results ↓ Assessment of title, abstract & keywords - by two independently working researchers
 (Thomas et al., 2004) ↓ 33 research publications @julauss #NODA16
15. MEDIA & DESIGN 15 @julauss #NODA16 Records identified from scientific databases (n= 772) Further publications from expert poll of data journalism researchers (n= 4) Excluded after screening (n= 739) Preliminary corpus: publications included after screening of records (n= 33) References from preliminary corpus (n = 1151) Final corpus: Publications included in the systematic review (n=40) Excluded after screening (n= 1148) Further publications included after screening of references (n = 3) Adapted from Fecher, Friesike & Hebing (2015)
16. MEDIA & DESIGN 16 @julauss #NODA16 software-assisted qualitative content analysis (Kaefer, Roper, & Sinha, 2015; Mayring, 2000; Schreier, 2012; QSR International, 2015) computational analysis of structural aspects
 (Kreibich, 2016; Lopez, 2009)
17. MEDIA & DESIGN Results
18. MEDIA & DESIGN Development of the literature over time n=40 @julauss 18 #NODA16
19. MEDIA & DESIGN 
 Publications by type and citations n=40 bubble size = number of citations in Google Scholar @julauss 19 #NODA16
20. MEDIA & DESIGN Affiliations & collaborations @julauss 20 #NODA16
21. MEDIA & DESIGN 1787-2015
 n=1644 @julauss 
 References per year 21 #NODA16
22. MEDIA & DESIGN Most-cited references Publication Nr. of Citations Meyer, P. (2002/1973). Precision journalism: A reporter’s introduction to social science methods (4th ed.). Oxford: Rowman & Littlefield. 15 Parasie, S., & Dagiral, E. (2013). Data-driven journalism and the public good: “Computer-assisted-reporters” and “programmer-journalists” in Chicago. New Media & Society, 15(6), 853–871. 15 Gray, J., Bounegru, L., & Chambers, L. (Eds.). (2012). The data journalism handbook: How journalists can use data to improve the news. Sebastopol: O’Reilly. 13 @julauss 22 #NODA16
23. MEDIA & DESIGN @julauss Theoretical frames • Science and technology studies • Actor network theory
 (Ausserhofer, 2015; De Maeyer, Libert, Domingo, Heinderyckx, & Le Cam, 2015; Parasie & Dagiral, 2013; Parasie, 2015)
 23 #NODA16
24. MEDIA & DESIGN @julauss Research designs & data collection methods Method Nr. of studies In-depth interviews 25 Content analysis 21 Survey 5 Short-term observation 3 Newsroom ethnography 1 Note. Content analysis includes analysis of news, databases, blogs, job ads, visualizations, briefings, manuals, and more. Short-term observation encompasses visits to the newsroom and participation in meetings. A newsroom ethnography is defined as a detailed study of a newsroom over the course of several days. 24 #NODA16
25. MEDIA & DESIGN Geographical scope Country Number of studies United States 16 United Kingdom 14 Germany 5 International 3 n/a 3 Sweden 2 Switzerland 2 Norway 2 Netherlands 2 … … @julauss 25 #NODA16
26. MEDIA & DESIGN @julauss 26 #NODA16 Research gaps in data journalism research • Comparision of practices between countries 
 (Appelgren & Nygren, 2014; Parasie & Dagiral, 2013) • Long-term studies (Davenport, 2000; Knight, 2015) • Newsroom ethnographies (Parasie & Dagiral, 2013) • Software studies (Garrison, 1999; Lewis, 2013; Stavelin, 2013) • Reader experience studies (Segel & Heer, 2010)
27. MEDIA & DESIGN Conclusion
28. MEDIA & DESIGN @julauss 28 #NODA16 • data journalism and its investigation has been developing rapidly • quality improvements in the research • issues with the literature: few publications refer to theory or methodology, just report what has been investigated Conclusion
29. MEDIA & DESIGN @julauss 29 #NODA16 • practices in small news organizations, freelancers, local and mobile data journalism etc. • gender • digital methods: investigating the field through its platforms • theory • … Research opportunities
30. MEDIA & DESIGN Explore the literature online at: http://literature.validproject.at Julian Ausserhofer1,2,3,
 julian.ausserhofer@fh-joanneum.at @julauss Robert Gutounig1,
 @sextus_empirico Michael Oppermann2,
 @oppermann_m Sarah Matiasek1,2 &
 @sarahmatiasek Eva Goldgruber1
 @evagoldgruber 1: FH Joanneum University of Applied Sciences, Graz
 2: University of Vienna
 3: Humboldt Institute for Internet and Society, Berlin NODA16 Academic Pre-Conference
 #NODA16
 21.04.2016, University of Helsinki
31. MEDIA & DESIGN References 31
32. MEDIA & DESIGN @julauss 32 #NODA16 Anderson, C. W. (2013). Towards a sociology of computational and algorithmic journalism. New Media & Society, 15(7), 1005–1021. doi: 10.1177/1461444812465137 Appelgren, E., & Nygren, G. (2014). Data journalism in Sweden: Introducing new methods and genres of journalism into “old” organizations. Digital Journalism, 2(3), 394–405. doi: 10.1080/21670811.2014.884344 Ausserhofer, J. (2015). „Die Methode liegt im Code”: Routinen und digitale Methoden im Datenjournalismus. In A. Maireder, J. Ausserhofer, C. Schumann, & M. Taddicken (Eds.), Digitale Methoden in der Kommunikationswissenschaft (pp. 87–111). Berlin: digitalcommunicationresearch.de. doi: 10.17174/dcr.v2.5 Davenport, L., Fico, F., & Detwiler, M. (2000). Computer–assisted reporting in Michigan daily newspapers: More than a decade of adoption. Presented at the Association for Education in Journalism and Mass Communication (AEJMC) National Convention, Phoenix, Arizona. De Maeyer, J., Libert, M., Domingo, D., Heinderyckx, F., & Le Cam, F. (2015). Waiting for data journalism: A qualitative assessment of the anecdotal take-up of data journalism in French-speaking Belgium. Digital Journalism, 3(3), 432–446. doi: 10.1080/21670811.2014.976415 Fecher, B., Friesike, S., & Hebing, M. (2015). What drives academic data sharing? PLoS ONE, 10(2), e0118053. doi: 10.1371/journal.pone.0118053 Fink, K., & Anderson, C. W. (2015). Data journalism in the United States: Beyond the “usual suspects.” Journalism Studies, 16(4), 467–481. doi: 10.1080/1461670X.2014.939852 Garrison, B. (1999). Newspaper size as a factor in use of computer-assisted reporting. Newspaper Research Journal, 20(3). Retrieved from http://com.miami.edu/car/baltimore1.htm Kaefer, F., Roper, J., & Sinha, P. (2015). A software-assisted qualitative content analysis of news articles: example and reflections. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 16(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/view/2123 Knight, M. (2015). Data journalism in the UK: A preliminary analysis of form and content. Journal of Media Practice, 16(1), 55–72. doi: 10.1080/14682753.2015.1015801 Kreibich, C. (2016). scholar.py. Retrieved from https://github.com/ckreibich/scholar.py
33. MEDIA & DESIGN @julauss 33 #NODA16 Lewis, S. C. (2015). Journalism in an era of big data. Digital Journalism, 3(3), 321–330. doi: 10.1080/21670811.2014.976399 Loosen, W., Reimer, J., & Schmidt, F. (2015). When data become news: A content analysis of data journalism pieces. Presented at the The Future of Journalism 2015 Conference, Cardiff. Lopez, P. (2009). GROBID: Combining automatic bibliographic data recognition and term extraction for scholarship publications. In M. Agosti, J. Borbinha, S. Kapidakis, C. Papatheodorou, & G. Tsakonas (Eds.), Research and Advanced Technology for Digital Libraries (pp. 473–474). Berlin: Springer. Retrieved from doi: 10.1007/978-3-642-04346-8_62 Massaro, M., Dumay, J. C., & Guthrie, J. (forthcoming). On the shoulders of giants: undertaking a structured literature review in accounting. Accounting, Auditing & Accountability Journal. Mayring, P. (2000). Qualitative content analysis. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 1(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/ view/1089 Parasie, S. (2015). Data-driven Revelation? Epistemological tensions in investigative journalism in the age of “big data.” Digital Journalism, 3(3), 364–380. doi: 10.1080/21670811.2014.976408 Parasie, S., & Dagiral, E. (2013). Data-driven journalism and the public good: “Computer-assisted- reporters” and “programmer-journalists” in Chicago. New Media & Society, 15(6), 853–871. doi: 10.1177/1461444812463345 QSR International. (2015). Nvivo. Retrieved from http://www.qsrinternational.com/ Schreier, M. (2012). Qualitative content analysis in practice. Thousand Oaks: SAGE. Segel, E., & Heer, J. (2010). Narrative visualization: Telling stories with data. IEEE Trans. Visualization and Computer Graphics, 16(6), 1139–1148. doi: 10.1109/TVCG.2010.179 Stavelin, E. (2013). Computational journalism: When journalism meets programming (Dissertation). University of Bergen, Bergen. Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14(3), 207–222. doi: 10.1111/1467-8551.00375