Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using RethinkDB and ArcGIS to create amazing ma...

Using RethinkDB and ArcGIS to create amazing map visualizations

In this talk, we'll take public data to create a visualization of restaurant density across the US. We'll use use RethinkDB to clean, analyze and store our data. Then, we'll feed the data into ArcGIS in order create an interactive visualization. After this talk, you'll get a sense of why a realtime database with geospatial features is an excellent tool for anything "geo" related.

Jorge Silva

June 02, 2015
Tweet

More Decks by Jorge Silva

Other Decks in Technology

Transcript

  1. What is RethinkDB? • Open source database for building realtime

    web applications • NoSQL database that stores schemaless JSON documents • Distributed database that is easy to scale
  2. Built for Realtime Apps • Subscribe to change notifications from

    database queries • No more polling — the database pushes changes to your app • Reduce the amount of plumbing needed to stream live updates
  3. RethinkDB Structure Database → Table → Document MySQL: Database →

    Table → Row MongoDB: Database → Collection → Document
  4. Sample Document { "name": "Will Riker", "position": "Commander", "height": 193,

    "birthdate": Mon Aug 19 2335, "ships": [ { "name": "USS Pegasus" }, { "name": "USS Potemkin" }, { "name": "USS Enterprise" }, ], ... }
  5. Introduction to ReQL • ReQL embeds natively into your programming

    language • Compose ReQL queries by chaining commands
  6. Additional ReQL Features • Date and time functions for time

    data • Support for storing binary objects • r.http allows you to execute http requests from the database
  7. Changefeeds r.table("users").changes() r.table("users") .insert({name: "Bob"}) Changefeed output: { new_val: {

    id: '362ae837-2e29-4695-adef-4fa415138f90', name: 'Bob', ... }, old_val: null }
  8. Changefeeds r.table("users").changes() r.table("users") .get("362ae837-2e29-4695-adef-4fa415138f90") .update({name: "Bobbby"}) Changefeed output: { new_val:

    { id: '362ae837-2e29-4695-adef-4fa415138f90', name: 'Bobby' }, old_val: { id: '362ae837-2e29-4695-adef-4fa415138f90', name: 'Bob' } }
  9. Introduction to Geospatial • Geospatial queries are integrated into ReQL

    • With changefeeds, you can listen to changes in your geospatial data
  10. Geospatial capabilities • Geometry types: point, line, polygon • Geometry

    Manipulation: fill, polygonSub • Non-Indexed Operations: distance, intersects, includes • Indexed operations: getNearest, getIntersecting
  11. Geospatial ReQL Query r.table("starbucks_locations") .getIntersecting(cityOfSacramento, { index: 'location'}) .orderBy(function (row)

    { return r.point(-122.482, 38.578) .distance(row('location')); }) Closest Starbucks to Hacker Lab in SAC
  12. Geospatial ReQL Query r.table("starbucks_locations") .getIntersecting(cityOfSacramento, { index: 'location'}) .orderBy(function (row)

    { return r.point(-122.482, 38.578) .distance(row('location')); }) Access the database table
  13. Geospatial ReQL Query r.table("starbucks_locations") .getIntersecting(cityOfSacramento, { index: 'location'}) .orderBy(function (row)

    { return r.point(-122.482, 38.578) .distance(row('location')); }) Get all points within polygon
  14. Geospatial ReQL Query r.table("starbucks_locations") .getIntersecting(cityOfSacramento, { index: 'location'}) .orderBy(function (row)

    { return r.point(-122.482, 38.578) .distance(row('location')); }) Order by distance to point
  15. Geospatial ReQL Query r.table("starbucks_locations") .getIntersecting(cityOfSacramento, { index: 'location'}) .orderBy(function (row)

    { return r.point(-122.482, 38.578) .distance(row('location')); }) Return the distance to a point
  16. Restaurants in county query return r.table("counties") // 1. Get number

    of restaurants .merge(...) // 2. Calculate restaurant density .merge(...) Get restaurant density
  17. Restaurants in county query Get all counties return r.table("counties") //

    1. Get number of restaurants .merge(...) // 2. Calculate restaurant density .merge(...)
  18. Restaurants in county query return r.table("counties") // 1. Get number

    of restaurants .merge({ restaurants: // .. }) // 2. Calculate restaurant density .merge(...) Merge a property into county
  19. Restaurants in county query return r.table("counties") // 1. Get number

    of restaurants .merge({ restaurants: r.table('restaurants') .getIntersecting(row('geometry'), { index: 'geometry' }) .count() }) // 2. Calculate restaurant density .merge(...) Get number of restaurants in county
  20. Restaurants in county query Get restaurants table return r.table("counties") //

    1. Get number of restaurants .merge({ restaurants: r.table('restaurants') .getIntersecting(row('geometry'), { index: 'geometry' }) .count() }) // 2. Calculate restaurant density .merge(...)
  21. Restaurants in county query Get intersecting geometries return r.table("counties") //

    1. Get number of restaurants .merge({ restaurants: r.table('restaurants') .getIntersecting(row('geometry'), { index: 'geometry' }) .count() }) // 2. Calculate restaurant density .merge(...)
  22. Restaurants in county query Count intersecting geometries return r.table("counties") //

    1. Get number of restaurants .merge({ restaurants: r.table('restaurants') .getIntersecting(row('geometry'), { index: 'geometry' }) .count() }) // 2. Calculate restaurant density .merge(...)
  23. Restaurants in county query return r.table("counties") // 1. Get number

    of restaurants .merge(...) // 2. Calculate restaurant density .merge({ stat: row('restaurants').div(row('area')) }) Add new `stat` property
  24. Restaurants in county query return r.table("counties") // 1. Get number

    of restaurants .merge(...) // 2. Calculate restaurant density .merge({ stat: row('restaurants').div(row('area')) }) Divide `restaurants` by `area`
  25. Client/ArcGIS • Data is queried and sent through a WebSocket

    connection • Polygon geometry is converted into ArcGIS Graphic and added to map • Color (Symbol) is applied dynamically based on max
  26. Summary • ReQL allows for complex geospatial queries • RethinkDB

    makes building realtime applications a lot easier