Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tracing Patterns of Growth and Maintenance in O...

Tracing Patterns of Growth and Maintenance in OpenStreetMap

Presentation at State of the Map US 2015

Alan McConchie

June 07, 2015
Tweet

More Decks by Alan McConchie

Other Decks in Research

Transcript

  1. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd Tracing Patterns of

    Growth and Maintenance in OpenStreetMap Alan McConchie Design Technologist, Stamen Design PhD Candidate, University of British Columbia [email protected] // @mappingmashups State of the Map US June 7, 2015 1
  2. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd Some questions after

    a decade of OpenStreetMap: 1. Do OSM's first “explorers” stick around? 2. Does activity shift to “gardening”? 3. Do imports hinder growth of community? 2
  3. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd Just last Saturday,

    I discovered that the edits I made in Aylmer and Hull — suburbs of Gatineau, Quebec that I know very well, because I pass through them every time I go to Ottawa — had been replaced by a batch import from CanVec. It removed a lot of human intelligence that I had added to the existing streets, such as pedestrian crossings, traffic lights, and turning circles. It also obliterated service roads, and turned all streets into highway=unclassified instead of residential, tertiary, or secondary, and divided highways into two- way streets. And any areas that had shared a node with a street way were now bollixed up. In other words, a big mess to revert, or to fix: the CanVec data had some useful information to add to the map — but not at the cost of erasing hours upon hours of existing work. My work. I was pissed[…] My basic point was, and is, the following: that you’re not going to get local people to contribute to OSM if they believe that their edits are going to be wiped out by the next person to import a pile of data. Jonathan Crowe, 2011 (emphasis mine) http://www.maproomblog.com/2011/02/the_state_of_openstreetmap_in_canada.php 11
  4. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd OSM is part

    of a greater movement of collaborative productivity, where people all over the world can and do join forces to create something great, something of value. I believe that in 40 years, probably even in 15, hardly anything of the data we have collected will retain much value - but we will have been part of a great development, and mankind will be the better for it. Will OSM, instead of being the social endeavor of “a great map that people made themselves”, then be the technical challenge of “the geo database where a few clever guys managed to combine lots of existing data”? Frederik Ramm, 2012 (emphasis mine) https://lists.openstreetmap.org/pipermail/talk-us/2012-December/009966.html 12
  5. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd A popular (although

    not universally accepted) theory is that massive imports are to blame for slowing down the development of the US community. …instead of the fun task of going out, finding new streets, and filling in a blank map, new contributors in the US were now faced with the relative tedium of correcting repetitive errors in an existing dataset. Meanwhile, in countries like Germany, almost all data has been collected by volunteer mappers, with only a few small-scale imports conducted in places where there already was an active community. This was not by design - there simply wasn't a comparable dataset that could have been imported - but it was ultimately beneficial. At least if you accept the theory, of course. Tobias Knerr, 2015 (emphasis mine) http://forum.openstreetmap.org/viewtopic.php?id=30121 13
  6. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd Community Imports •

    Divide data into chunks • Community members manually upload each chunk NOTE: The term "Import" is highly loaded in the OSM community. "A distributed and curated merge," is a more accurate description of what Seattle OSM planning to do. — wiki.osm.org/wiki/Seattle_Import 15
  7. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd Building community through

    imports Four training events and editathons Five events out walking around 20+ active participants slideshare.net/gwhathistory/osm-sotm-us-2013-imports4community-002 16
  8. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd NYC community building

    import • Estimated 1500 hours of work • Volunteers + Mapbox employees • Coordination with NYC GIS Dept animation: mapbox.com/blog/nyc-buildings-openstreetmap 17
  9. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 19 Did TIGER

    effect how local OSM communities grow and evolve? Do the recent imports have different data signatures from TIGER? Broadly, how has OSM evolved differently in different cities?
  10. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 20 How important

    is that moment of filling-in the blank spots on the map?
  11. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd • Complete  history

     dump  of  OSM  up  to  December  31,  2014   • Extract  a  small  number  of  study  areas   • Only  look  at  nodes,  disregard  all  other  data  types   • Overlay  grid  (1  km)   • Find  earliest  node  in  each  grid  cell   • For  each  username,  count  following  values:   Number  of  total  node  edits     Number  of  version  1  edits  (node  creation)   Number  of  version  2+  edits  (move  node  or  change  attributes)   Number  of  edits  in  “blank  spots  on  the  map”   • Only  count  a  user's  activity  within  each  study  area.   21 The  analysis
  12. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 25 Downtown  Vancouver

       1000m  grid   First  OSM  nodes  in  each  cell
  13. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 27 Greater  London:

      OSM  contributors  plotted  by   number  of  “blank  spot”  edits   and  number  of  total  edits.   Circles  are  individuals  editors   sized  according  to  number  of   days  active   London
  14. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 28 San  Francisco

     Bay  Area:   OSM  contributors  plotted  by   number  of  “blank  spot”  edits   and  number  of  total  edits.   Circles  are  individuals  editors   sized  according  to  number  of   days  active   Bay Area
  15. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 29 Selected bounding

    boxes around approximate extent of urban areas Bounding box population estimated using Gridded Population of the World dataset for year 2000 (GPWv3) London Population 8,500,000 Land area 3300 sq km Berlin Population 1,900,000 Land area 2100 sq km Seattle Population 3,200,000 Land area 6300 sq km
  16. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 30 Total edited

    nodes by month (new nodes + modified nodes) TIGER imports Boston building import Haiti earthquake Paris building import second Boston import NYC building imports seattle building import
  17. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 31 Total modified

    nodes by month Modification history lost before OSM API v0.5 Berlin Haiti Berlin Bay Area Montevideo London New York Bay Area Los Angeles Paris Los Angeles
  18. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 33 Total edited

    nodes by month normalized by area Boston Boston Amsterdam Amsterdam Paris
  19. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 34 Total edited

    nodes by month normalized by population Boston Boston Amsterdam Montevideo
  20. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 35 Total edited

    nodes by month: London (new nodes + modified nodes)
  21. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 39 Created nodes

    by month: London red: blankspot contributors light blue: non-blankspot contributors
  22. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 40 Modified nodes

    by month: London red: blankspot contributors light blue: non-blankspot contributors
  23. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 41 i Total

    created nodes by month: London solid line: aggregated blankspot contributors (n = 130) dotted line: aggregated non-blankspot contributors (n = 5488)
  24. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 42 Total modified

    nodes by month: London solid line: aggregated blankspot contributors (n = 130) dotted line: aggregated non-blankspot contributors (n = 5488)
  25. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 43 Total created

    nodes by month: Berlin solid line: aggregated blankspot contributors (n = 109) dotted blue: aggregated non-blankspot contributors (n = 6893)
  26. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 44 Total modified

    nodes by month: Berlin solid line: aggregated blankspot contributors (n = 109) dotted blue: aggregated non-blankspot contributors (n = 6893)
  27. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 45 Total created

    nodes by month: Toronto solid line: aggregated blankspot contributors (n = 58) dotted line: aggregated non-blankspot contributors (n = 1395)
  28. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 46 Total modified

    nodes by month: Toronto solid line: aggregated blankspot contributors (n = 58) dotted line: aggregated non-blankspot contributors (n = 1395)
  29. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 47 Total created

    nodes by month: Vancouver solid line: aggregated blankspot contributors (n = 72) dotted line: aggregated non-blankspot contributors (n = 1004)
  30. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 48 Total modified

    nodes by month: Vancouver solid line: aggregated blankspot contributors (n = 72) dotted line: aggregated non-blankspot contributors (n = 1004)
  31. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 49 Total modified

    nodes by month: Haiti solid line: aggregated blankspot contributors (n = 317) dotted line: aggregated non-blankspot contributors (n = 1120)
  32. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 50 Total modified

    nodes by month: Haiti solid line: aggregated blankspot contributors (n = 317) dotted line: aggregated non-blankspot contributors (n = 1120)
  33. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 51 Total created

    nodes by month: Four U.S. cities solid lines: aggregated blankspot contributors (Bay Area n = 119, Seattle = 54, New York = 61, Boston = 27) dotted lines: aggregated non-blankspot contributors (Bay Area n = 3485, Seattle = 2067, New York = 3116, Boston = 1406)
  34. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd 52 Total modified

    nodes by month: Four U.S. cities solid lines: aggregated blankspot contributors (Bay Area n = 119, Seattle = 54, New York = 61, Boston = 27) dotted lines: aggregated non-blankspot contributors (Bay Area n = 3485, Seattle = 2067, New York = 3116, Boston = 1406)
  35. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd Discussion: • Power

    law effects trump everything else. • Gradual shift to new contributors (less so in Canada!) • Subtle shift to maintenance tasks… • …but there are always new things to add. • Maintenance is “bursty”, like additions are. • Do imports hinder the growth of community? 53
  36. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd Discussion: • Power

    law effects trump everything else. • Gradual shift to new contributors (less so in Canada!) • Subtle shift to maintenance tasks… • …but there are always new things to add. • Maintenance is “bursty”, like additions are. • Do imports hinder the growth of community? 54 …not very clear cut at all
  37. [email protected] // @mappingmashups // #sotmus // sta.mn/mkd @mappingmashups [email protected] 55

    http://almccon.github.io/mapgardening/timeseries.html (work in progress: use at own risk!) Thanks!