News Production Workflows in Data-driven, Algorithmic Journalism: A Systematic Literature Review

Ausserhofer, J., Gutounig, R., & Oppermann, M. (2015, October). News Production Workflows in Data-driven, Algorithmic Journalism: A Systematic Literature Review. Presented at the Dubrovnik Media Days, Dubrovnik.

In the past decade, the “experimental use of algorithms, data and social science methods” (Gynnild, 2014, p. 715) in journalism has been growing rapidly. What was known as Computer-­assisted reporting and database journalism and what was mainly employed by election analysts, weather reporters, and investigative journalists in the United States (Cox, 2000), is today important for many more practitioners in all kinds of newsrooms and departments all over the world. Data­-driven journalism has become a key theme in journalism education, in the public discourse about journalism, and in journalism research. But in what way have journalistic workflows and routines actually changed due to the broad introduction of data in the newsroom? How has this impacted journalistic norms and ethics, and the skills requirements for journalists?
Based on a systematic review of scholarly papers and industry reports this paper develops a comprehensive conceptual model for newsroom processes in data­driven journalism. It discusses regional differences, the background and epistemology of data journalists, and enabling and hindering factors of data-­driven news projects. In the conclusion, the article identifies gaps in the research landscape that are fruitful for future investigations.

­­--
Cox, M. (2000). The Development of Computer­Assisted Reporting. Presented at the Newspaper Division,
Association for Education in Journalism and Mass Communication, Southeast Colloquium, Chapel Hill,
North Carolina.
Gynnild, A. (2014). Journalism Innovation Leads to Innovation Journalism: The Impact of Computational
Exploration on Changing Mindsets. J ournalism, 1 5( 6), 713–730. http://doi.org/10.1177/1464884913486393

Acknowledgements
The research for this paper is part of the project V isual Analytics in Data-driven Journalism (VALID) t hat is supported by a grant of the A ustrian Ministry for Transport, Innovation and Technology (BMVIT). VALID is financed under the I CT of the Future funding programme of the A ustrian Research Promotion Agency (FFG). Project number: 845598.

Transcript

1. MEDIA & DESIGN News Production Workflows in Data- driven, Algorithmic Journalism A Systematic Literature Review Julian Ausserhofer1,2 , @julauss Robert Gutounig1 & @sextus_empirico Michael Oppermann2 @oppermann_m 1: FH Joanneum University of Applied Sciences Graz 2: University of Vienna Dubrovnik Media Days, 31.10.15 #DuMD2015 & #DUMediaDays
2. MEDIA & DESIGN *Tool evaluation partners * * * Supported by:
3. MEDIA & DESIGN Research Interest
4. MEDIA & DESIGN #DuMD2015 & #DUMediaDays @julauss, @sextus_empirico & @oppermann_m 4 "experimental use of algorithms, data and social science methods" in journalism (Gynnild, 2014, p. 715)
5. MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 5 (Cox, 2000) http://www.computerhistory.org/revolution/early-computer-companies/5/102/445 (Meyer, 1973) 1952 #DuMD2015 & #DUMediaDays Computational methods in journalism
6. MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 6 (Garrison, 1996) #DuMD2015 & #DUMediaDays
7. MEDIA & DESIGN The datafication of journalism @julauss, @sextus_empirico & @oppermann_m 7 "datafication" of society (Mayer-Schönberger & Cukier, 2013) "increased focus on measurement, outcomes assessment" (Anderson, 2015, p. 363) "journalism's quantitative turn" (Coddington, 2015) … has also affected the "infrastructures of journalism" (Russ-Mohl, 2006, p. 199) #DuMD2015 & #DUMediaDays
8. MEDIA & DESIGN Research literature on data journalism @julauss, @sextus_empirico & @oppermann_m 8 "internalist tendencies at [... the] early stage of academic research" (Anderson, 2013, p. 1007) ↓ "an explosion in data journalism-oriented scholarship" (Fink & Anderson, 2015, p. 476)* "rapidly growing body" of scientific studies (Lewis, 2015, p. 322)* *cited via (Loosen, Reimer & Schmidt, 2015, p. 2) #DuMD2015 & #DUMediaDays
9. MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 9 In what way have journalistic workflows and routines changed due to the broad introduction of data in the newsroom? How has this impacted journalistic norms and ethics, and the skill requirements for journalists? #DuMD2015 & #DUMediaDays Research questions
10. MEDIA & DESIGN Method
11. MEDIA & DESIGN #DuMD2015 & #DUMediaDays @julauss, @sextus_empirico & @oppermann_m 11 Systematic literature review
12. MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 12 "to develop insights, critical reflections, future research paths and research questions" (Massaro, Dumay & Guthrie, forthcoming) It adopts "a replicable, scientific and transparent process [...] that aims to minimize bias [...]" (Tranfield, Denyer & Smart, 2003) #DuMD2015 & #DUMediaDays Why a systematic literature review?
13. MEDIA & DESIGN Undertaking a systematic literature review 13 Adapted from Massaro et al. (forthcoming) Own visualization @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Writing a literature review protocol Developing insights and critique through analyzing the dataset Developing future research paths and questions Determining the type of studies and carrying out a comprehensive literature search Coding data Defining the questions that the literature review should answer
14. MEDIA & DESIGN 14 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Determining the type of studies and carrying out a comprehensive literature search Defining the questions that the literature review should answer In what way have journalistic workflows and routines changed due to the broad introduction of data in the newsroom? How has this impacted journalistic norms and ethics, and the skill requirements for journalists?
15. MEDIA & DESIGN 15 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensive literature search • Empirical research on DDJ • Social science focus, but open to other disciplines • Published after 1995 Included Journal articles Book sections Conference papers Reports (from industry and research projects) PhD theses Not included Bachelor's and Master's theses Press reports Blog posts
16. MEDIA & DESIGN 16 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensive literature search • Preliminary search with “data-driven journalism” • Extracting related terms from the keyword section of research papers
17. MEDIA & DESIGN 17 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensive literature search Search terms algorithmic journalism computational journalism computer-assisted reporting data journalism data-driven journalism data-driven reporting database journalism datajournalism datenjournalismus quantitative journalism No search terms accountability journalism crowdsourced journalism dataviz datavis ddj drone journalism investigative journalism online journalism open journalism
18. MEDIA & DESIGN 18 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensive literature search Scientific Databases ACM Digital Sowiport EBSCO Springer IEEE SpringerLink JSTOR Taylor & Francis Online ProQuest Web of Science Science Direct Wiley Scopus Google Scholar Sociological Abstracts
19. MEDIA & DESIGN 19 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensive literature search 772 search results ↓ Assessment of title, abstract & keywords - by two independently working researchers (Thomas et al., 2004) ↓ 33 research publications
20. MEDIA & DESIGN 20 Determining the type of studies and carrying out a comprehensive literature search Defining the questions that the literature review should answer Developing future research paths and questions Developing insights and critique through analyzing the dataset Coding data @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays qualitative coding computational analysis
21. MEDIA & DESIGN Results
22. MEDIA & DESIGN Literature Corpus
23. MEDIA & DESIGN 23 Research literature on workflows in DDJ Development of literature over time n=33 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays
24. MEDIA & DESIGN 24 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Research literature on workflows in DDJ Publication type Parasie & Dagiral, 2013 Lewis & Usher, 2013 Diakopolus, 2015 Karlsen & Stavelin, 2014 n=33 node size = number of citations in Google Scholar
25. MEDIA & DESIGN 25 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Research literature on workflows in DDJ Publication year of referenced papers 1787-2015 n=1151 Constitution of the United States
26. MEDIA & DESIGN 26 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Research literature on workflows in DDJ Publication year of referenced papers 1999-2015 n=1004 2013: 176 cited papers
27. MEDIA & DESIGN 27 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Research literature on workflows in DDJ Most-cited references
28. MEDIA & DESIGN 28 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays edges = 165 corpus publications, n=33 references, n = 70 size = nr. of citations from the corpus Research literature on workflows in DDJ Corpus and most-cited references The Journalist as Programmer Precision Journalism Accountability through Algorithm
29. MEDIA & DESIGN Nr. of studies Survey 3 In-depth interviews 22 (Particpant) observation 4 Method(s) of data collection 29 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays
30. MEDIA & DESIGN Geographical scope 30 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Nr. of studies Nr. of studies United States 9 Multinational (>4) 2 United Kingdom 9 Norway 1 Germany 4 Netherlands 1 Sweden 2 Argentina 1 Switzerland 2 Austria 1 Finland 2 Belgium 1
31. MEDIA & DESIGN Qualitative analysis - News Production Workflow
32. MEDIA & DESIGN News production workflow Collecting Collecting digitalized information from • publicly available data sets • requested data • own research or data collected through crowdsourcing • automated research (scraping) • leaked data (17; Loosen, 2015) 32 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays
33. MEDIA & DESIGN Type of data source • government or public bureau • private corporation • other organizations like NGOs, research institutes (28, Loosen, 2015) 33 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Collecting
34. MEDIA & DESIGN Access to information in Norway and USA generally considered good (due to freedom of information legislation) Barriers to easy data access • fees • lacking competencies for exporting data at public offices • the form data is often delivered (e.g. as PDF) • lack of legal legal resources to get access to data that is denied to journalists in the first place (6; 14) 34 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Collecting
35. MEDIA & DESIGN Selecting data Because data journalists don’t have much time in the newsroom, they prefer easy-accessible data, e.g. datasets from public bodies. They are very cautious when it comes to work with data from companies or private sources. (14) 35 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Collecting
36. MEDIA & DESIGN Methods • Journalists use statistical programs (e.g. MySQL, Excel, Access, SPSS, Google Docs, Maps, Google Refine, Google Fusion Tables etc.) • Statistical methods and packages including cluster, network & regression analysis --> Journalists are quite cautious with it, since they consider this outside of their field of expertise. (6, 14) 36 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Analyzing
37. MEDIA & DESIGN Results of DDJ research depend in increasing manner on (graphical) display of (automated) data analysis software Tools and programming languages • Developers use Python, Ruby on Rails, JavaScript, HTML • Used third-party software includes MySQL, Access, Excel, Caspio, Tableau, Arcview, TimelineSetter etc. (14, 17, 28) 37 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Building and visualizing
38. MEDIA & DESIGN DDJ is connected to a special way of representation which consists of • displaying key messages as graphics and/or • building interactive web applications Storytelling is done through (1) interactive web applications with continuous update of data (2) interactive application based on a stable data set (3) continuously updated info graphic or (4) static infographics (17) 38 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Building and visualizing
39. MEDIA & DESIGN • Working alone at small and medium-sized newspapers, and part of teams at larger news organizations • New connections between journalists and programmers • Members of teams usually have complementary backgrounds (6, 9, 14, 18) 39 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Collaboration
40. MEDIA & DESIGN • Rise of "annotative journalism" • Regarding the collaboration between newsroom and ICT- department in media corporations data journalists perceive the relation as troublesome. Rather data journalists try to bypass ICT department when installing extra software or setting up databases or servers (6, 28) 40 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Collaboration
41. MEDIA & DESIGN • The goal of DDJ activities is by definition the [very accurate] journalistic publication • Publication of data sources and raw data following open data principles is mentioned too as an integral part of DDJ. (6, 17, Coddington, 2014) 41 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Publishing the product
42. MEDIA & DESIGN Backgrounds of data journalists
43. MEDIA & DESIGN Backgrounds of data journalists • Data journalists are appx. 5 years younger than the average age of German journalists • All of them have started a university program, 80% have also finished it • There is no generalizable “data journalism” career path. Background is mostly either social science or informatics. • Backgrounds include programming, design, typography, info graphics, usability, databases, Web and journalism • USA: dominance of National Institute for Computer-Assisted Reporting (NICAR) (14, 17, 18; 6) 43 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays
44. MEDIA & DESIGN Enabling and Hindering Factors
45. MEDIA & DESIGN Enabling and hindering factors Critical factors • Skills of the involved journalists • Support networks (internal and external) important for the development of skills • Availability of data • Tools • Legal resources • Available resources (6, 14, 17) 45 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays
46. MEDIA & DESIGN Conclusion
47. MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 47 #DuMD2015 & #DUMediaDays • New kind of people arriving in the newsroom • New skills needed in the newsroom • Increase of collaborative newswork • Shift to data-driven approaches (and epistemologies debates connected to that)
48. MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 48 Scope: • Practices outside of Europe and North America • The construction of "facts" • Gender aspects Methods of data collection: • Survey • (Participant) observation • Content analysis • Digital methods #DuMD2015 & #DUMediaDays Research gaps
49. MEDIA & DESIGN 49 Explore the (preliminary) corpus online – it's a protoype: http://tinyurl.com/valid-survey http://www.validproject.at Julian Ausserhofer1,2 , @julauss julian.ausserhofer@fh-joanneum.at Robert Gutounig1 & @sextus_empirico robert.gutounig@fh-joanneum.at Michael Oppermann2 @oppermann_m michael.oppermann@univie.ac.at 1: FH Joanneum University of Applied Sciences Graz 2: University of Vienna Dubrovnik Media Days, 31.10.15 #DuMD2015 & #DUMediaDays
50. MEDIA & DESIGN Tools & Literature 50
51. MEDIA & DESIGN 51 NVIVO [Computer Software]. (2015). Retrieved from http://www.qsrinternational.com/ Bostock, M., Ogievetsky, V., & Heer, J. (2011). D³ data-driven documents.Visualization and Computer Graphics, IEEE Transactions on, 17(12), 2301-2309. Lopez, P. (2009). GROBID: Combining automatic bibliographic data recognition and term extraction for scholarship publications. In Research and Advanced Technology for Digital Libraries (pp. 473-474). Berlin: Springer. Zotero [Computer Software]. (2015). Retrieved from http://zotero.org/ Tools & software libraries @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays
52. MEDIA & DESIGN 52 Anderson, C. W. (2013). Towards a Sociology of Computational and Algorithmic Journalism. New Media & Society, 15(7), 1005–1021. http://doi.org/10.1177/1461444812465137 Anderson, C. W. (2015). Between the Unique and the Pattern: Historical Tensions in Our Understanding of Quantitative Journalism. Digital Journalism, 3(3), 349–363. http://doi.org/10.1080/21670811.2014.976407 Cox, M. (2000). The Development of Computer-assisted Reporting. Presented at the Newspaper Division, Association for Education in Journalism and Mass Communication, Southeast Colloquium, Chapel Hill, North Carolina. Fink, K., & Anderson, C. W. (2015). Data Journalism in the United States: Beyond the "Usual Suspects". Journalism Studies, 16(4), 467–481. http://doi.org/10.1080/1461670X.2014.939852 Garrison, B. (1996, September). Computer-Assisted Reporting: A National Use Update. Presented at the National Society of Professional Journalists Meeting, Washington, D.C. Lewis, S. C. (2015). Journalism In An Era Of Big Data. Digital Journalism, 3(3), 321–330. http://doi.org/10. 1080/21670811.2014.976399 Loosen, W., Reimer, J., & Schmidt, F. (2015). When Data Become News: A Content Analysis of Data Journalism Pieces. Presented at the The Future of Journalism 2015 Conference, Cardiff. Mayer-Schönberger, V., & Cukier, K. (2013). Big Data: A Revolution That Will Transform How We Live, Work, and Think. Boston: Houghton Mifflin Harcourt. Primary Sources @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays
53. MEDIA & DESIGN 53 Massaro, M., Dumay, J. C., & Guthrie, J. (forthcoming). On the Shoulders of Giants: Undertaking a Structured Literature Review in Accounting. Accounting, Auditing & Accountability Journal. Meyer, P. (1973). Precision Journalism: A Reporter’s Introduction to Social Science Methods (2nd ed.). Oxford: Rowman & Littlefield. Russ-Mohl, S. (2006). The Economics of Journalism and the Challenge to Improve Journalism Quality: A Research Manifesto. Studies in Communication Sciences, 6(2), 189–208. Thomas, J., Harden, A., Oakley, A., Oliver, S., Sutcliffe, K., Rees, R., … Kavanagh, J. (2004). Integrating Qualitative Research with Trials in Systematic Reviews. BMJ : British Medical Journal, 328(7446), 1010–1012. http://doi.org/10.1136/bmj.328.7446.1010 Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a Methodology for Developing Evidence-Informed Management Knowledge by Means of Systematic Review. British Journal of Management, 14(3), 207–222. http://doi.org/10.1111/1467-8551.00375 Primary Sources @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays
54. MEDIA & DESIGN 54 (34) Aitamurto, T., Sirkkunen, E., & Lehtonen, P. (2011). Trends in Data Journalism. Retrieved from http://virtual.vtt.fi/virtual/nextmedia/Deliverables-2011/D3. 2.1.2.B_Hyperlocal_Trends_In%20Data_Journalism.pdf (2) Ananny, M., & Crawford, K. (2015). A Liminal Press: Situating news app designers within a field of networked news production. Digital Journalism, 3(2), 192–208. http://doi.org/10.1080/21670811.2014.922322 (12) Appelgren, E., & Nygren, G. (2012). Data Journalism in Sweden – Opportunities and Challenges: A Case Study of Brottspejl at Sveriges Television (SVT). In C. Mills, M. Pidd, & E. Ward (Eds.), Proceedings of the Digital Humanities Congress 2012. Sheffield: HRI Online Publications. Retrieved from http://www. hrionline.ac.uk/openbook/chapter/dhc2012-appelgren (11) Appelgren, E., & Nygren, G. (2014). Data Journalism in Sweden: Introducing New Methods and Genres of Journalism into “Old” Organizations. Digital Journalism, 2(3), 394–405. http://doi.org/10.1080/21670811.2014.884344 (18) Ausserhofer, J. (2015). „Die Methode liegt im Code”: Routinen und digitale Methoden im Datenjournalismus. In A. Maireder, J. Ausserhofer, C. Schumann, & M. Taddicken (Eds.), Digitale Methoden in der Kommunikationswissenschaft (Vol. 2, pp. 87–111). Berlin: digitalcommunicationresearch.de. Retrieved from http://nbn-resolving.de/http://dx.doi.org/10.17174/dcr.v2.5 (3) Baack, S. (2013). A New Style of News Reporting: Wikileaks and Data-driven Journalism. In B. Rambatan & J. Johanssen (Eds.), Cyborg Subjects: Discourses on Digital Culture (pp. 113–122). CreateSpace Independent Publishing. Retrieved from http://www.ssoar.info/ssoar/handle/document/40025 (26) Bakker, P. (2014). Mr. Gates Returns: Curation, Community Management and Other New Roles for Journalists. Journalism Studies, 15(5), 596–606. http: //doi.org/10.1080/1461670X.2014.901783 (31) Daniel, A., & Flew, T. (2010). The Guardian Reportage of the UK MP Expenses Scandal: A Case Study of Computational Journalism. In Record of the Communications Policy and Research Forum 2010 (pp. 186–194). Sydney: Network Insight. Retrieved from http://eprints.qut.edu.au/39358/ (35) De Maeyer, J., Libert, M., Domingo, D., Heinderyckx, F., & Le Cam, F. (2015). Waiting for Data Journalism: A Qualitative Assessment of the Anecdotal Take- up of Data Journalism in French-Speaking Belgium. Digital Journalism, 3(3), 432–446. http://doi.org/10.1080/21670811.2014.976415 (1) Diakopoulos, N. (2015). Algorithmic Accountability: Journalistic Investigation of Computational Power Structures. Digital Journalism, 3(3), 398–415. http: //doi.org/10.1080/21670811.2014.976411 (21) Dick, M. (2014). Interactive Infographics and News Values. Digital Journalism, 2(4), 490–506. http://doi.org/10.1080/21670811.2013.841368 (19) Felle, T. (2015). Digital Watchdogs? Data Reporting and the News Media’s Traditional “Fourth Estate” Function. Journalism, 1464884915593246. http: //doi.org/10.1177/1464884915593246 Corpus 1/3 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays
55. MEDIA & DESIGN 55 (14) Fink, K., & Anderson, C. W. (2015). Data Journalism in the United States: Beyond the “Usual Suspects.” Journalism Studies, 16(4), 467–481. http://doi. org/10.1080/1461670X.2014.939852 (27) Garrison, B. (1999). Newspaper Size as a Factor in Use of Computer-Assisted Reporting. Newspaper Research Journal, 20(3). Retrieved from http://com. miami.edu/car/baltimore1.htm (22) Gynnild, A. (2014). Journalism Innovation Leads to Innovation Journalism: The Impact of Computational Exploration on Changing Mindsets. Journalism, 15 (6), 713–730. http://doi.org/10.1177/1464884913486393 (7) Hannaford, L. (2015). Computational Journalism in the UK Newsroom: Hybrids or Specialists? Journalism Education, 4(1), 6–21. http://doi.org/http://hdl. handle.net/2173/575953 (30) Howard, A. B. (2014). The Art and Science of Data-Driven Journalism (pp. 1–144). New York: Tow Center for Digital Journalism. Retrieved from http: //towcenter.org/wp-content/uploads/2014/05/Tow-Center-Data-Driven-Journalism.pdf (8) Hullman, J., Diakopoulos, N., Momeni, E., & Adar, E. (2015). Content, Context, and Critique: Commenting on a Data Visualization Blog. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (pp. 1170–1175). New York: ACM. http://doi.org/10.1145/2675133. 2675207 (6) Karlsen, J., & Stavelin, E. (2014). Computational Journalism in Norwegian Newsrooms. Journalism Practice, 8(1), 34–48. http://doi.org/10.1080/17512786. 2013.813190 (13) Knight, M. (2015). Data Journalism in the UK: A Preliminary Analysis of Form and Content. Journal of Media Practice, 16(1), 55–72. http://doi.org/10. 1080/14682753.2015.1015801 (33) Koch, C., & Stollorz, V. (2014). Transparenz oder Konfusion? Methodik und empirische Basis des Datenjournalismus in der Medizin. In V. Lilienthal, D. Reineck, & T. Schnedler (Eds.), Qualität im Gesundheitsjournalismus (pp. 367–387). Wiesbaden: Springer Fachmedien. Retrieved from http://link.springer. com/chapter/10.1007/978-3-658-02427-7_21 (28) Lewis, S. C., & Usher, N. (2013). Open Source and Journalism: Toward New Frameworks for Imagining News Innovation. Media, Culture and Society, 35(5), 602–619. http://doi.org/10.1177/0163443713485494 (5) Lewis, S. C., & Usher, N. (2014). Code, Collaboration, and the Future of Journalism: A Case Study of the Hacks/Hackers Global Network. Digital Journalism, 2 (3), 383–393. http://doi.org/10.1080/21670811.2014.895504 (29) Lugo-Ocando, J., & Brandão, R. F. (2015). Stabbing News: Articulating Crime Statistics in the Newsroom. Journalism Practice, 9, 1–15. http://doi.org/10. 1080/17512786.2015.1058179 Corpus 2/3 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays
56. MEDIA & DESIGN 56 (10) Parasie, S. (2015). Data-driven Revelation? Epistemological Tensions in Investigative Journalism in the Age of “Big Data.” Digital Journalism, 3(3), 364–380. http://doi.org/10.1080/21670811.2014.976408 (9) Parasie, S., & Dagiral, E. (2013). Data-driven Journalism and the Public Good: “Computer-assisted-reporters” and “Programmer-journalists” in Chicago. New Media & Society, 15(6), 853–871. http://doi.org/http://dx.doi.org/10.1177/1461444812463345 (32) Radchenko, I., & Sakoyan, A. (2014). The View on Open Data and Data Journalism: Cases, Educational Resources and Current Trends. In D. I. Ignatov, M. Y. Khachay, A. Panchenko, N. Konstantinova, & R. E. Yavorsky (Eds.), Analysis of Images, Social Networks and Texts (pp. 47–54). Cham: Springer International Publishing. Retrieved from http://link.springer.com/chapter/10.1007/978-3-319-12580-0_4 (25) Uskali, T., & Kuutti, H. (2015). Models and Streams of Data Journalism. The Journal of Media Innovations, 2(1), 77. http://doi.org/10.5617/jmi.v2i1.882 (24) Veglis, A. (2005). Modeling Computer Supported Collaborative Work in a Newspaper Editorial Process. In Proceedings of the 9th WSEAS International Conference on Computers (pp. 1–6). Stevens Point: World Scientific and Engineering Academy and Society (WSEAS). Retrieved from http://dl.acm. org/citation.cfm?id=1369599.1369726 (16) Weber, W., & Rall, H. (2012). Data Visualization in Online Journalism and Its Implications for the Production Process. In 2012 16th International Conference on Information Visualisation (IV) (pp. 349–356). Montpellier: IEEE. http://doi.org/10.1109/IV.2012.65 (36) Weber, W., & Rall, H. (2013). “We Are Journalists”: Production Practices, Attitudes and a Case Study of the New York Times Newsroom. In W. Weber, M. Burmester, & R. Tille (Eds.), Interaktive Infografiken (pp. 161–172). Berlin: Springer. Retrieved from http://link.springer.com/chapter/10.1007/978-3-642- 15453-9_9 (17) Weinacht, S., & Spiller, R. (2014). Datenjournalismus in Deutschland: Eine explorative Untersuchung zu Rollenbildern von Datenjournalisten. Publizistik, 59 (4), 411–433. http://doi.org/10.1007/s11616-014-0213-5 (20) Young, M. L., & Hermida, A. (2015). From Mr. and Mrs. Outlier to Central Tendencies: Computational Journalism and Crime Reporting at the Los Angeles Times. Digital Journalism, 3(3), 381–397. http://doi.org/10.1080/21670811.2014.976409 Corpus 3/3 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays
57. MEDIA & DESIGN 57 Explore the (preliminary) corpus online – it's a protoype: http://tinyurl.com/valid-survey http://validproject.at Julian Ausserhofer1,2 , @julauss julian.ausserhofer@fh-joanneum.at Robert Gutounig1 & @sextus_empirico robert.gutounig@fh-joanneum.at Michael Oppermann2 @oppermann_m michael.oppermann@univie.ac.at 1: FH Joanneum University of Applied Sciences Graz 2: University of Vienna Dubrovnik Media Days, 31.10.15 #DuMD2015 & #DUMediaDays