Upgrade to Pro — share decks privately, control downloads, hide ads and more …

From Big Data to Mobile Data with Apache CouchD...

From Big Data to Mobile Data with Apache CouchDB and PouchDB at Apache: Big Data

It’s all too easy for mobile app developers to assume that their apps will run on fast and reliable networks. The reality for end users, though, is often slow, unreliable networks with spotty coverage. What happens when the network doesn’t work, or when a device is in airplane mode? You get unhappy, frustrated users. One solution is to take an offline-first approach. An offline-first app is an app that works, without error, when there is no network connection. Offline-first apps built with Apache CouchDB and PouchDB (an open source JavaScript database) can provide better, faster user experiences by storing data locally and then synchronizing with a cloud database when a network connection is available.

Bradley Holt

May 10, 2016
Tweet

More Decks by Bradley Holt

Other Decks in Programming

Transcript

  1. Apache: Big Data Bradley Holt, Developer Advocate Tuesday, May 10,

    2016 From Big Data to Mobile Data with Apache CouchDB and PouchDB @BradleyHolt
  2. Mobile First Design for the smallest device first and then

    apply progressive enhancement techniques to take advantage of larger screen sizes when available
  3. Offline First Design for offline usage first and then apply

    progressive enhancement techniques to take advantage of network connectivity when available
  4. The Eight Fallacies of Distributed Computing 1.  The network is

    reliable 2.  Latency is zero 3.  Bandwidth is infinite 4.  The network is secure 5.  Topology doesn't change 6.  There is one administrator 7.  Transport cost is zero 8.  The network is homogeneous @BradleyHolt
  5. JSON Documents {
 _id: "6EF9D2B0-13D3-1378-8D30-39E3CE0B36C2",
 _rev: "1-0b457efcf82fb29492ef927ba5b6ee15",
 type: "Feature",
 geometry:

    {
 type: "Point",
 coordinates: [
 -71.1028,
 42.3691
 ]
 },
 properties: {
 session_id: "3486b13f-7b8a-8a96-dfbf-9b82800e367f",
 timestamp: 1422928591717
 }
 } @BradleyHolt
  6. Creating a New Document var db = new PouchDB("smart-meter"); db.put({

    _id: "2014-11-12T23:27:03.794Z", kilowatt_hours: 14 }).then(function() { console.log("Document created"); }).catch(function(error) { console.log(error); }); @BradleyHolt
  7. Updating a Document db.put({ _id: "2014-11-12T23:27:03.794Z", kilowatt_hours: 14 }).then(function(response) {

    return db.get(response.id); }).then(function(doc) { // Update the value for kilowatt hours doc.kilowatt_hours = 15; // Put the document back to the database return db.put(doc); }).catch(function(error) { console.log(error); }); @BradleyHolt
  8. Querying a Database with allDocs db.bulkDocs([ {_id: "2014-11-12T23:27:03.794Z", kilowatt_hours: 14},

    {_id: "2014-11-13T00:52:01.471Z", kilowatt_hours: 15}, {_id: "2014-11-13T01:39:28.911Z", kilowatt_hours: 16}, {_id: "2014-11-13T02:52:01.471Z", kilowatt_hours: 17} ]).then(function(result) { // Get all documents return db.allDocs({include_docs: true}); }).then(function(response) { console.log(response); }).catch(function(error) { console.log(error); }); @BradleyHolt
  9. allDocs Options §  include_docs – conflicts – attachments §  startkey §  endkey

    §  inclusive_end (true by default) §  limit §  skip §  descending §  key §  keys @BradleyHolt
  10. Querying a Database with Map/Reduce §  Most queries can be

    done with allDocs (in PouchDB) §  Map functions transform documents into indexes §  Reduce functions aggregate results of Map functions – _sum – _count – _stats @BradleyHolt
  11. Querying a Database with PouchDB Find §  Based on Cloudant

    Query (Mango) §  Declarative indexes §  MongoDB-style query language @BradleyHolt
  12. PouchDB Data Storage Limits @BradleyHolt Firefox Chrome Opera 15+ Internet

    Exporer 10+ iOS Safari Safari (desktop) Android PhoneGap / Cordova Data Storage Limit 50MB (more with user permission) calculated calculated 250MB (prompts user at 10 MB) 50MB (prompts user at 5MB and at increments) unlimited (prompts user at 5MB and at increments) calculated / 200MB unlimited Adapter IndexedDB IndexedDB / WebSQL IndexedDB / WebSQL IndexedDB WebSQL WebSQL IndexedDB / WebSQL SQLite
  13. Apache CouchDB CouchDB is a document database featuring an HTTP

    API, JSON documents, and peer-to-peer replication @BradleyHolt
  14. Bidirectional Replication db.sync(remoteDb, { live: false, retry: false }).on("change", function(info)

    { // Replication has written a new document console.log(info); }).on("complete", function(info) { // Replication has complete or been cancelled console.log(info); }).on("error", function(error) { // Replication has stopped due to an unrecoverable failure console.log(error); }); @BradleyHolt
  15. Live Replication var sync = db.sync(remoteDb, { live: true, retry:

    true }).on("change", function(info) { // Replication has written a new document console.log(info); }).on("complete", function(info) { // Replication has complete or been cancelled console.log(info); }).on("error", function(error) { // Replication has stopped due to an unrecoverable failure console.log(error); }); @BradleyHolt
  16. Filtered Replication db.replicate.to(remoteDb, { filter: function(doc) { return doc._id >=

    "2014-11-13T00:00:00.000Z"; } }).on("change", function(info) { // Replication has written a new document console.log(info); }).on("complete", function(info) { // Replication has complete or been cancelled console.log(info); }); @BradleyHolt
  17. One Database Per User @BradleyHolt userdb-d76846 userdb-905cec userdb-adc95b userdb-c082f2 userdb-730bba

    userdb-c3d3e5 userdb-a1ec1f userdb-85a327 userdb-9b9aba userdb-85bcfe rdb-da3d25
  18. Boilerplates & Tools §  Frontend Web Apps –  React Boilerplate

    with Service Workers <https://github.com/mbrio/react-boilerplate/tree/react-0.13-flummox-service> §  Backend Web Apps –  PouchDB npm Package <https://www.npmjs.com/package/pouchdb> –  PouchDB Server npm Package <https://www.npmjs.com/package/pouchdb-server> §  Mobile Apps –  PouchDB for Ionic Framework <https://github.com/nolanlawson/pouchdb-ionic> –  "Hello world" Cordova app with PouchDB <https://github.com/nolanlawson/pouchdb-cordova-hello-world> –  "Hello world" Cordova app with PouchDB, using the SQLite Plugin <https://github.com/nolanlawson/pouchdb-cordova-hello-world-with-sqlite-plugin> –  Cloudant FoodTracker (uses Cloudant Sync for iOS) <https://github.com/ibm-cds-labs/cloudant-food-tracker> §  Desktop Apps –  PouchDB for Electron (formerly Atom Shell) <https://github.com/nolanlawson/pouchdb-electron> –  PouchDB for Chrome packaged apps <https://github.com/nolanlawson/pouchdb-chrome-app> –  "Hello world" Chrome app with PouchDB <https://github.com/nolanlawson/pouchdb-chrome-app-hello-world> –  PouchDB for NW.js (aka Node-Webkit) <https://github.com/nolanlawson/pouchdb-nw> §  Internet of Things (IoT) Apps –  Node-RED <http://nodered.org/> @BradleyHolt
  19. Image Credits §  Open for Data Dome (outside) by Bradley

    Holt §  Open for Data Dome (inside) by Bradley Holt §  Brent Spiner (Data from Star Trek: The Next Generation) with Zoltar from Big by Bradley Holt, on Twitter <https://twitter.com/BradleyHolt/status/702311271002087424> §  A mockup of the golden Apple iPhone 5S by Zach Vega, on Wikimedia Commons <https://commons.wikimedia.org/wiki/File:IPhone_5s.png> §  Joan Touzet (@wohali), ASF Member, CouchDB PMC Member <https://twitter.com/wohali/status/595689720933445632> §  Device landscape by Jeremy Keith, on Flickr <https://flic.kr/p/anLcHu> §  Cloud Formation Over the Adirondacks by Bradley Holt, on Twitter <https://twitter.com/BradleyHolt/status/623030107679002624> §  Cell phone tower by Gary Lerude, on Flickr <https://flic.kr/p/crL7TN> §  Pneumatic Central by Sleestak, on Flickr <https://flic.kr/p/mRvRQ> §  Colunas by Daniel Zanini H., on Flickr <https://flic.kr/p/5ZwHWv> §  Speed DLR on Doklands by Umberto Rotundo, on Flickr <https://flic.kr/p/7GmcUo> §  Waterfall by Paulo Valdivieso, on Flickr <https://flic.kr/p/oNkvRP> §  Wildfire by U.S. Fish and Wildlife Service Southeast Region, on Flickr <https://flic.kr/p/8zkWGd> §  Arduino Uno by Pete Prodoehl, on Flickr <https://flic.kr/p/a3ky7E> §  Mango with section on a white background by bangdoll, on Flickr <https://flic.kr/p/9CBP2h> §  Warning by Stefano Brivio, on Flickr <https://flic.kr/p/tuBHA> @BradleyHolt