Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Offline storage & automatic data synchronizatio...

Offline storage & automatic data synchronization for your web and mobile applications

More and more web and mobile developers are using local storage to ensure good response times and that applications continue to work in the absence of a network connection. But how do you sync local storage with the backend? How do you resolve conflicts? In this talk, I present to you an open source framework (Amplify DataStore) that allows you to develop modern web and mobile applications that answer these questions with a minimum of a line of code.

More Decks by Sébastien Stormacq - AWS Developer Advocate

Other Decks in Technology

Transcript

  1. OFFLINE FIRST Core features should function with or without an

    internet connection. WHAT DOES IT MEAN? #01
  2. OFFLINE FIRST Data is written to and queried locally from

    the end user’s device and periodically uploaded and replicated in the database. WHAT DOES IT MEAN? #02
  3. OFFLINE FIRST Provide end users with a consistent UX when

    internet connectivity is slow or non-existent. WHAT DOES IT MEAN? #03
  4. REAL TIME An application that functions within a time frame

    that the user senses as immediate or very close to it. WHAT DOES IT MEAN #01
  5. REAL TIME Cohesive system where data is in sync across

    all connected devices. WHAT DOES IT MEAN #02
  6. MENTAL MODEL struct Episode: Identifiable, Hashable, Codable { let id:

    String let date: String let title: String let duration: String let description: String? } let newEpisode = Episode(id: UUID().uuidString, date: Date(), title: “New Episode", duration: “00:12:14”, description: "Description")
  7. Storage Engine Storage Adapter CRUD OBSERVES WRITES READS WRITES OBSERVES

    Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer
  8. DATA MODELING WITH GRAPHQL Developer AUTHORING BUILD codegen typescript, java,

    kotlin, swift type EpisodeData { id: ID! date: String! title: String! duration: String! description: String } struct Episode { let id: String let date: String let title: String let duration: String let description: String? }
  9. DATASTORE Fluent interface Mutations / writes let episode = new

    Episode(date: "14 nov 2022", title: "New Episode", duration: "New Duration") try await Amplify.DataStore.save(episode)
  10. Storage Engine Storage Adapter CRUD OBSERVES WRITES READS ARCHITECTURE DataStore

    API Developer Model converted to / from the storage adapter of choice Object and action persistence with specific storage engine (SQL Lite, IndexDB, etc..)
  11. Storage Engine Storage Adapter CRUD OBSERVES WRITES READS OBSERVES WRITES

    / READS Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer Model converted to GraphQL statement Response converted to Model and written to storage engine.
  12. LOCAL STORE + SUBSCRIPTIONS Fluent interface - observing data models

    let e = EpisodeData.keys Amplify.DataStore .observeQuery( for: EpisodeData.self, where: e.podcastDataEpisodesId == podcast.id)
  13. SINGLE REHYDRATE QUERY let e = EpisodeData.keys Amplify.DataStore .observeQuery( for:

    EpisodeData.self, where: e.podcastDataEpisodesId == podcast.id)
  14. Provide a way to merge concurrent modi fi cations, always,

    in any order. CONFLICT DETECTION & RESOLUTION
  15. person = { name: “Jennifer”, age: 32 } person =

    { name: “Jennifer”, age: 32, title: “Developer”, } person = { name: “Jennifer”, age: 32, title: “Engineer” } original item client a client b CONFLICT RESOLUTION - AUTO MERGE
  16. person = { name: “Jennifer”, age: 32, title: “Developer” }

    person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION - AUTO MERGE person = { name: “Jennifer”, age: 32, title: “Developer” } new item
  17. person = { name: “Jennifer”, age: 32 } person =

    { name: “Jennifer”, age: 32, location: “Nebraska”, } person = { name: “Jennifer”, age: 32, title: “Engineer” } original item client a client b CONFLICT RESOLUTION - AUTO MERGE
  18. person = { name: “Jennifer”, age: 32, location: “Nebraska” }

    person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION - AUTO MERGE person = { name: “Jennifer”, age: 32, location: “Nebraska”, title: “Engineer” } new item
  19. person = { name: “Jennifer”, age: 32 hobbies: [“climbing"] }

    person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “reading”] } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “karaoke”] } original item client a client b CONFLICT RESOLUTION - AUTO MERGE Lists
  20. person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “reading”]

    } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”,“karaoke”] } client a client b CONFLICT RESOLUTION - AUTO MERGE Lists person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “karaoke”, “reading”] } new item
  21. AMPLIFY DATASTORE CONFLICT DETECTION & RESOLUTION • Monotonic counters •

    Controlled by the backend (AppSync) • GraphQL types with versions used for merge or reject decisions • Base table & change table with soft deletes & TTL
  22. MONOTONIC COUNTER person = { id: 100, age: 30, _version:

    1 } person = { id: 100, age: 31, _version: 2 } person = { id: 100, age: 32, _version: 3 } id age name _version 100 32 Amy 3 101 28 Charles 9 102 47 Melissa 8 Base Table
  23. ds_pk ds_sk age name _version _lastChangedAt Person-9386:2020-01-23 13:16:08.127:9386:1 32 Amy

    1 1579785368127 Person-9386:2020-01-23 13:16:21.785:9386:2 33 Amy 2 1579785381785 Person-9386:2020-01-23 13:16:37.859:9386:3 34 Amy 3 1579785397859 Change Table typename + uuid + date timestamp + uuid + version CONFLICT RESOLUTION
  24. person = { age: 34, age: 35, location: “Seattle” _version:

    4 } User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE ONLINE person = { age: 34, location: “Seattle” _version: 3 }
  25. User A User B person = { age: 32, _version:

    1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE ONLINE person = { age: 34, location: “Seattle” _version: 3 } person = { age: 34, location: “Seattle” _version: 4 } person = { age: 35, location: “Seattle” _version: 4 }
  26. AUTO MERGE SYNC ENABLED GRAPHQL RESOLVERS Base table + change

    table + GraphQL types with versions used for merge or reject decisions
  27. id age name _version 100 32 Amy 8 101 28

    Charles 9 DELETE ID 102 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3
  28. ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James

    7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC syncPerson(lastSync: 13:16:22.127) Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859 Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859
  29. id age name _version _deleted _ttl 100 32 Amy 8

    101 28 Charles 9 102 47 Melissa 3 TRUE 1582384003 SYNCHRONIZATION ds_pk ds_sk age _deleted _ttl Person-9386:2020-01-23 13:16:08.127:9386:1 32 1582384003 Person-9386:2020-01-23 13:16:21.785:9386:2 33 1582384003 Person-9386:2020-01-23 13:16:37.859:9386:3 47 TRUE 1582384003 Change Table Base Table
  30. TIME TO LIVE ds_pk ds_sk age _deleted _ttl Person-9386:2020-01-23 13:16:08.127:9386:1

    32 1582384003 Person-9386:2020-01-23 13:16:21.785:9386:2 33 1582384003 Person-9386:2020-01-23 13:16:37.859:9386:3 47 TRUE 1582384003 1582384003 1582384003 1582384003