Core Member of Hood.ie - Organizer of OpenTechSchool Dortmund ! twitter@misprintedtype github@zoepage Hello, my name is…Thank you very much for your time and for having me. And also a huge thank you for the sponsors. ! I’m super excited to be here!
4. Implementation @misprintedtype Today I’ll tell you: ! 1. What the offline first concept is and why we need it in our applications so badly. 2. Where the problems occur on the client and the server side. 3. How we, as developers have to rethink our application logic 4. And I will show you one way to implement the concept ! So, let’s start! :)
the internet changed. A lot. ! We do different things, with different devices than 25, 10, 5 years ago... and it is a part of our everyday life! ! Who used their phone today for more than 5 different tasks today? More than 10? More than 15? ! We rely on our devices... the internet, but our needs changed.
work online on a train with a hotspot? On a plane? How well did that go? ! Who of you ever did a screenshot of an app, because they were scared of losing their data? ! Who of you ever stood in a supermarket and try to ask someone what they should buy, but realized you are offline? ! What if your traffic limit is reached? Your Hotel WIFI is way too expensive or the roaming costs are nuts? ! Those are issues we all know even from the wealthy countries many of us live in. !
on being online. If you are not online, it seems like you do not exist! ! We need new ways to deal with these issues and be able to stay a part of the online community, stay in touch with our friends and family.
countries with poor or even no network coverage. ! Zero reception, unreliable and inconsistent connections are still an issue, no matter where we go. ! And our experiences and discussions with people from all over the globe have shown that this is a global problem which won't just solve itself globally in the near future. ! We need to accept, the technology has reached its limits.
PERMANENT, FAST CONNECTIVITY, WHERE A TEMPORARY DISCONNECTION OR SLOW SERVICE IS REGARDED AS A PROBLEM AND COMMUNICATED AS AN ERROR.” @misprintedtype -> drink ! The applications we use and build for other people… We have to think this topic through and they need to be built offering solutions. ! Go away from the desktop mindset of permanent, fast connectivity by accepting being offline is not a bug, but a reoccuring state. ! Let’s start and design applications with keeping offline first in mind!
out there! ! Starting with email apps, which send the emails we write, for us when we are online again. ! Writing down notes during meetings with http://minutes.io or creating all lists we need with https://createlists.net/ help us stay productive without sending error messages as soon as our internet connection is lost. ! Sending pictures, tweets, messages during a concert can be tricky when everyone is doing it in this exact moment! applications like twitter store the message and it’s at least not lost. Great! Why not sending it, when you are online? Whatsapp sync works here pretty well! ! I see amazing apps in the future! ! Imagine… You are in an area you’ve never been before? GREAT! I’m sure, you’d love to explore that, right? Wouldn’t it be great if your map would realize where you are and just download the map for you when you have WIFi and so make it offline available? Showing you the sightseeing points, the next cafe with wifi, the nearby ATM… Help you to feel like almost at home!
strategy to help cope with the unknowable circumstances and connection states our users may find themselves in. We know almost nothing about them, and can take almost nothing for granted. ! Offline First is simply an acknowledgement that this lack of certainty extends a bit further than we previously thought.
interfaces mobile-first, it’s time to think about designing and building our applications offline-first. ! With an offline mode that isn’t only a contingency for an error scenario, but a fundamentally more flexible and fault-tolerant way of application design. ! So, what are the main points we have to think about, when we create the concepts for offline first applications?
Does the app even need to inform the human of the current connection state? Is this information even relevant to the human? If so, how can this best be done? !
trust in the app, leaving them safe in the knowledge that it won’t forget the last state it was in, and that it won’t lose any data, regardless of what the connectivity situation is? So they do not need to think about making screenshots of their apps anymore? ! ! 3. How can I communicate to the human that creating data within the app is still possible offline, and that it will be posted/dispatched/properly dealt with in the future? !
the icon in the upper right corner. ! The user types something, the state changes to unchanged. Then it gets updated and it is saved again automatically. ! This really looks like a good practice.
that occur in the users current view when they reconnect and the server pushes new and changed data? ! What to do with deleted items, things that can’t be organised in lists, objects that aren’t in themselves immutable (like emails)? ! 5. How can I make the unavoidable resolution of conflicts by the human as painless and intuitive as possible? !
any prefered decisions on the part of the human and pre-load any data they might need later? What metrics (possibly behavioural) are there to base these decisions on? ! ! How can I word all of these scenarios in a way that doesn’t worry the users, that embraces offline as a reality without causing problems? ! This are the issues from the user side we have to think about… ! What does that mean for the server behavior?
of truth! The client was always just the prophet… It communicates always with the server, sender requests, waited for answers and told him everything what happened. If the server is not available… the client panicked and was helpless, didn’t know what to do!
out a request and waited for the respond, the data was stored and the view rendered and reloaded again. ! With an async request, the client at least was on call and didn’t need to wait all the time for the server… a step forward, but still… it was on call. ! Kind of a really unhealthy, overdependent relationship we know at least from really bad movies.
and so has the internet. It empowered us and so it also empowered the client! ! So, with offline first… we want to decouple this relationship. ! But how shall we deal with the new states? What ARE the new states?
offline and the push will fail right away… what then? Storing the draft? Making a queue and trying to send it again, after 1, 2, 5, 10 seconds? Blocking the requests? Showing an error or just ignoring it and make it disappear, like it never happened?
the client, but not all of it! ! It can be tricky to find out the current connection state, independent if this is a native or a web app. ! So, should the app show the user just half of the data? Show nothing? Show as much as possible and trying again to resync? ! Can the user change the data? And how can we keep his frustration level as low as possible? !
to account for the fact that the request failed immediately, or after a timeout, or only a partial response was received. ! Timeouts need to be chosen in a way that is useful for the end user, a common default is 1 minute and users generally don’t wait that long. Or would you? ! And always remember… when it’s more that 1 second, keep the user busy. Preload images, load data step by step if it is available… and so on! ! In case of a timeout, there should be a user interface affordance for cancelling an operation, so that the user can apply a shorter timeout if that makes sense in the situation.
starting to show the top half of a web page, or email, or other message. ! The time it takes the user to read through that might be enough to fetch the rest of the data. Sometimes not enough data is provided to do anything at all and the user interface should account for that.
offers you a whole backend, with user accounts, storage, shares, email… it’s offline first including a localStorage and automatic sync. It has a dreamcode API, every frontend dev understands in a second. ! Hoodie abstracts away all the boring boilerplate backend stuff you don’t want to think about. ! When you check the hoodie website… don’t be surprised, because it looks different. This is a sneak peak just for you… we’ll launch it next week. ! Let’s take a look at the architecture.
of your choice! ! This, by itself, is enough for an app! ! Always think about… Do you even need to store anything on the server-side apart from credentials? You might not! Your business model might even profit from not doing it. ! But we want to check out the sync… so...
the database. ! We can do this because each user has their own private database which they only can access. You can’t share your data by accident! It’s private by default. ! So it’s very simple to decide what gets synced and how to handle changed data... ! You, of course, can share or publish it, if you want!
! -> Explain a direct message moving through the system. ! 1.You create a message in your app and send it to the hoodie layer. 2.The hoodie layer sends out a task to the localStorage, so the message can be stored locally and resolves with a promise. At the same time, it sends a task to the sync layer. 3.When the device is online, sync layer pushes the message to the CouchDB. It get’s stored and tasks are send out to other devices to update their client, because we changed the data. 4.And the plugin gets the task to send the message. ! ! The main point here is, the client and the server, they never have to talk directly to each other. They only leave each other messages and tasks, ! it’s all very loosely coupled and event-based. Which means it can be interrupted at any stage without breaking. ! Messages and tasks will be delivered and acted upon whenever possible. !
care whether the backend is actually there or not. ! It will hang on to your data and sync whenever it can AND if your UI allows it, users can keep working without interruption!
try to talk to the server directly, only through sync, and you always want to keep a local copy in your browser. ! And sync is hard! Really! Don’t even think about implementing it yourself, there is just too much to go wrong. ! But you can use a lot of stuff, which is around.
manifest. ! -> Who is already using it in their applications? ! The manifest gives you the possibility to fetch the files you’d love to load from the cache, when you are offline. ! So the app will load and work correctly if you hit the reload button. ! You can still interact with your application on the client side and store data, if you use an in-browser storage like local-/sessionStorage or IndexedDB.
experience for users @misprintedtype The advantages are… ! - You can use the app offline ! - No trip to the network, so the loading is fast ! - Your site is down? No worries! Your users will get the full (offline) experience
FF 3.5+ / Chrome 4.0+ / Safari 4.0+ / Opera 10.5+ / iPhone 2.0+ / Android 2.0+ @misprintedtype To use the Manifest, we need to do 3 steps! ! 1. including the manifest in our HTML files.
* FALLBACK: /online.html /offline.html ! @misprintedtype And the 3... You need actually a manifest… ! Let’s take a look at it! ! You always need to have the first line! CACHE: specify all files, which needs to be cached. No matter if this is an absolute or a relative path. NETWORK: define all resources, which can just be used online FALLBACK: define fallbacks
(also if you are online!) @misprintedtype The disadvantages are… ! 1. When you visit a site, the files always come straight from the cache! After the rendering, the browser will look for updates to the manifest and cached files. You need to reload the page to get the new files…
changed @misprintedtype The Appcache only updates, if the manifest changed. ! HTTP already has a caching model. Each file can define how it should be cached. Even at a basic level, individual files can say “never cache me,” or “check with the server, it’ll tell you if there’s an update,” or “assume I’m good until 1st April 2022.” ! As a slightly unusual workaround, the browser will only look for updates to files listed in the manifest if the manifest file itself has changed since the browser last checked. Any change that makes the manifest byte-for-byte different will do.
@misprintedtype THE APPLICATIONCACHE IS AN ADDITIONAL CACHE, NOT AT ALTERNATIVE ONE ! When the browser updates the ApplicationCache, it requests urls as it usually would. It obeys regular caching instructions: ! If an item’s header says “assume I’m good until 1st April 2022” the browser will assume that resource is indeed good until 1st April 2022, and won’t trouble the server for an update. ! Without specifics, the browser will take a guess at the caching. ! All files you serve should have cache headers and this is especially important for everything in your manifest and the manifest itself. If a file is very likely to update, it should be served with no-cache.
You might think you can treat your manifest as a static file, as in “assume I’m good until 1st April 2022,” then change the url to the manifest when you need to make updates. ! Don’t do that! Ever! ! When the user visits a page a second time they’ll get the ApplicationCached version. If you’ve changed the url to the manifest, bad luck, the user’s cached version still points at the old manifest, which is obviously byte-for-byte the same so nothing updates, ever.
NON-CACHED RESOURCES WILL NOT LOAD ON A CACHED PAGE ! If you cache index.html but not the background image, that image will not display on index.html, even if you’re online. ! ! And these are just the simple problems…
is a solution for that! ! Gregor Martynus tought the App Cache some manners and developed the app cache nanny. It helps us to deal with most of the problems the app cache still has, using an iframe for the manifest. So the files are served from the server, when you are online, it checks if the manifest is up-to-date. ! And there is also another silver lining!
It’s not done yet, but will be by the end of the year for Chrome and Firefox… and they are talking to apple, so hopefully in Safari soon too. It will be a very big thing next year, I think. ! I don’t want to overdo my time today, so I won’t talk about Service Worker, but you should watch Jake Archibalds talk from JSConfEU this year. And Jake is really fun and an amazing guy, his talks are legendary. ! There is the link.
@misprintedtype First law of Offline First is do not lose data! ! While decoupled the client and server side, we really need to store it on both sides.
@misprintedtype There for we trust PouchDB for the client and CouchDB on the server side. They are in a deep love relationship and a perfect team to make offline first quick and easy.
we want for sync. ! CouchDB comes with a suite of features, such as on-the-fly document transformation and real-time change notifications, we use for our message and task system. ! It also has a very basic version control system which helps you to manage your data. So when you data is out of sync, it’s making automatic intelligent decisions and helps you to merge the data. !
is designed to run well within the browser and store all your data locally while offline. ! Per default it uses IndexedDB in Chrome and Firefox and WebSQL in Safari to store the data. You also can add adapters for local and session storage. ! It sync’s with CouchDB and compatible servers, which speak the CouchDB replication protocol, like Cloudant and the Couchbase Sync Gateaway… when the application is back online. … It also works on mobile devices and for native apps e.g. with phonegap, cordova and many more…
every user has his own database and it’s private by default. So we solve a lot of problems from scratch. ! When we have one database per user, we always know where to sync to. When the version of the document isn’t up-to-date, we can check by opaque document revision id which to sync first and how to merge the data without real conflicts. ! Opaque document revision id is an md5 hash over the current doc version and list of history. ! When we receive partial data, we store the version of the document and can grab that before merging. ! Thanks to the event listeners and task, we can synchronize the data across devices… this means multi device usage on-the-fly.
data (replicated) You want to share your data? Sure! ! When we want to share some of our private data, we can „publish“ it. So it get’s replicated and others are able to read / read + manipulate the data. ! Then we are able to sync it again in our application logic into our private storage.
you should be able to switch between a local database or an online CouchDB instance without changing any of your application's code. ! However, there are some minor differences to note: ! View Collation: CouchDB uses ICU to order keys in a view query; in PouchDB they are ASCII ordered. ! View Offset: CouchDB returns an offset property in the view results. In PouchDB, offset just mirrors the skip parameter rather than returning a true offset. ! Please keep that in mind.
Chrome / Opera / Android 4.4+ IndexedDB % of storage y IE 10+ SQLite 250MB n Mobile Safari WebSQL 50MB n Sarafi WebSQL 5MB -> 500MB y Android 4.3 and lower IndexedDB 200MB n A question I get asked a lot is… how much data can PouchDB store? ! In Firefox, PouchDB uses IndexedDB, which will ask the user if data can be stored the first time it is attempted, then every 50MB after. The amount that can be stored is unlimited. ! Chrome calculates the amount of available storage on the user's hard drive and uses that to calculate a limit. ! Opera should work the same as Chrome, since it's based on Blink. ! Internet Explorer 10+ has a hard 250MB limit. ! Mobile Safari on iOS has a hard 50MB limit, while desktop Safari will prompt users wanting to store more than 5MB up to a limit of 500MB. ! Android works the same as Chrome as of 4.4+, while older version can store up to 200MB. ! In PhoneGap/Cordova, you can have unlimited data on both iOS and Android by using the SQLite Plugin.
the concept of offline first and how to build applications that handle offline first not as a bug, but as a feature! ! If you’d like to say hi, have any questions or want to grab some hoodie stickers, I’ll be around all day and also at the after party.
zoepage
elenatanasoiu
mackee
ioki
handlename
nrslib
broadleaf
yug1224
contour_gara
musan
hirokunimaeta
koic
k1mu21
ipullrank
akademiya2063
jlugia
raygrieselhuber
baasie
palkan
stuffmc
reverentgeek
cassininazir
ryanjones
