livedb-rethinkdb: Building collaborative apps with ShareJS and RethinkDB

Transcript

1. livedb-rethinkdb: Building collaborative apps with ShareJS and RethinkDB Jorge Silva · @thejsj · Developer Evangelist @ RethinkDB
2. Overview What is Operational Transformation? ShareJS: A Node.js library for OT RethinkDB: A database for realtime apps livedb-rethinkdb: RethinkDB + ShareJS
3. Text here...
4. Text here... Text here... User #1 User #2
5. Text here... Text here... User #1 User #2 How could these two users collaborate?
6. Every time a user makes an edit, push an updated document
7. abc User #1 User #2 ‘abc’ Server abc
8. abc User #1 User #2 ‘abc’ ‘abc’ abc Server abc
9. abc User #1 User #2 ‘xyz’ xyz 1ms later: Server xyz ‘abc’
10. xyz User #1 User #2 ‘abc’ ‘xyz’ xyz ‘abc’ ‘xyz’ Server xyz
11. This system doesn’t take latency into consideration
12. What if, instead of sending a completely updated document, we just sent the user’s actions?
13. User #1 User #2 abc { op: [1, ‘bc’] } a Server abc
14. abc User #1 User #2 abc { op: [1, ‘bc’] } { op: [1, ‘bc’] } Server abc
15. User #1 User #2 { op: [3, ‘efg’] } 012abc 1ms later: { op: [0, ‘012’ ] } abcefg Server 012efgabc
16. 012efgabc User #1 User #2 012abcefg { op: [3, ‘efg’] } { op: [0, ‘012’ ] } Server 012efgabc
17. This strategy is not able to accomodate the user’s intentions User #2’s changes are not inserted at the right cursor position.
18. Merely sending operations to the server doesn’t solve our problem. We need to tranform our operations.
19. What is Operational Transformation?
20. A class of algorithms that do multi-site realtime concurrency. Gives you eventual consistency - between multiple users - without retries - without errors - without any data being overwritten http://sharejs.org/
21. User #1 User #2 { op: [3, ‘def’] } 012abc 1ms later: { op: [0, ‘012’] } abcdef Server 012abcdef { op: [0, ‘012’] } { op: [6, ‘def’] } Transformed into: Transformed into:
22. 012abc User #1 User #2 012abcdef Server 012abcdef { op: [0, ‘012’] } Not Applied Transformed into: { op: [0, ‘012’] } { op: [0, ‘012’] }
23. 012abcdef User #1 User #2 abcdef Server 012abcdef { op: [6, ‘def’] } { op: [6, ‘def’] } Transformed into: { op: [6, ‘def’] } Not Applied
24. Possible applications Collaborative text and code editors Realtime wikis Any type of concurent text editing Examples: Google Docs, hackpad, etc.
25. OT is an important tool for building realtime applications
26. How can I use this in my own projects?
27. Enter ShareJS “ShareJS is an Operational Transform library for NodeJS & browsers. It lets you easily do live concurrent editing in your app.”
28. Demo http://livedb-rethinkdb.thejsj.com/
29. // WebSocket Connection var ws = new WebSocket(‘ws://localhost:8005’); var shareJS = new window.sharejs.Connection(ws); // Textarea var textareaDoc = shareJS.get(‘documents’, ‘helloworld’); textareaDoc.subscribe(); // Wait for document to be ready textareaDoc.whenReady(function () { setTimeout(function () { if (textareaDoc.type && textareaDoc.type.name === ‘text’) { // Attach textarea to document var elem = document.getElementById(‘helloworld’); textareaDoc.attachTextarea(elem); } }); }); ShareJS on the client https://github.com/thejsj/sharejs-rethinkdb-example/blob/master/client/index.html#L102-L124
30. // WebSocket Connection var ws = new WebSocket(‘ws://localhost:8005’); var shareJS = new window.sharejs.Connection(ws); // Textarea var textareaDoc = shareJS.get(‘documents’, ‘helloworld’); textareaDoc.subscribe(); // Wait for document to be ready textareaDoc.whenReady(function () { setTimeout(function () { if (textareaDoc.type && textareaDoc.type.name === ‘text’) { // Attach textarea to document var elem = document.getElementById(‘helloworld’); textareaDoc.attachTextarea(elem); } }); }); ShareJS on the client https://github.com/thejsj/sharejs-rethinkdb-example/blob/master/client/index.html#L102-L124
31. var sharejs = require(‘share’); var livedb = require(‘livedb’); // Connect to the database var db = require(‘livedb-rethinkdb’)({ host: ‘localhost’, port: 28015, db: ‘sharejs’ }); var backend = livedb.client(db); // Attach livedb instances to ShareJS var share = sharejs.server.createClient({ backend: backend }); ShareJS on the server: database https://github.com/thejsj/sharejs-rethinkdb-example/blob/master/server/livedb-client.js
32. var sharejs = require(‘share’); var livedb = require(‘livedb’); // Connect to the database var db = require(‘livedb-rethinkdb’)({ host: ‘localhost’, port: 28015, db: ‘sharejs’ }); var backend = livedb.client(db); // Attach livedb instances to ShareJS var share = sharejs.server.createClient({ backend: backend }); ShareJS on the server: database https://github.com/thejsj/sharejs-rethinkdb-example/blob/master/server/livedb-client.js
33. var wss = new WebSocketServer(); var Duplex = require(‘stream’).Duplex; var share = sharejs.server.createClient({ backend: backend }); // On socket connection wss.on(‘connection’, function () { var stream = new Duplex({ objectMode: true }); stream._write = function (chunk, encoding, callback) { client.send(JSON.stringify(chunk)); return callback(); }; client.on(‘message’, function (data) { return stream.push(JSON.parse(data)); }); return share.listen(stream); }); ShareJS on the server https://github.com/thejsj/sharejs-rethinkdb-example/blob/master/server/socket-handler.js#L8-L49
34. var wss = new WebSocketServer(); var Duplex = require(‘stream’).Duplex; var share = sharejs.server.createClient({ backend: backend }); // On socket connection wss.on(‘connection’, function () { var stream = new Duplex({ objectMode: true }); stream._write = function (chunk, encoding, callback) { client.send(JSON.stringify(chunk)); return callback(); }; client.on(‘message’, function (data) { return stream.push(JSON.parse(data)); }); return share.listen(stream); }); ShareJS on the server https://github.com/thejsj/sharejs-rethinkdb-example/blob/master/server/socket-handler.js#L8-L49
35. Open source database for building realtime applications NoSQL database that stores schemaless JSON documents Distributed database that is easy to scale What is RethinkDB?
36. Subscribe to change notifications from database queries No more polling – the database pushes changes to your app Reduce the amount of plumbing needed to stream live updates Built for realtime
37. var r = require(‘rethinkdb’); r .db(‘share_js’) .table(‘documents’) .filter({ name: ‘helloworld’ }) .run(conn) .then(function (cursor) { // Go through every row returned by the query cursor.each(function (row) { console.log(row); }); }); Changefeeds
38. var r = require(‘rethinkdb’); r .db(‘share_js’) .table(‘documents’) .filter({ name: ‘helloworld’ }) .run(conn) .then(function (cursor) { // Go through every row returned by the query cursor.each(function (row) { console.log(row); }); }); Changefeeds
39. var r = require(‘rethinkdb’); r .db(‘share_js’) .table(‘documents’) .filter({ name: ‘helloworld’ }) .changes() .run(conn) .then(function (cursor) { // Gets fired every time a row changes cursor.each(function (row) { console.log(row); }); }); Changefeeds
40. ShareJS & RethinkDB npm install sharejs livedb livedb-rethinkdb --save
41. Access documents
42. Access every single operation
43. Listen for new operations
44. Try it! https://github.com/thejsj/sharejs-rethinkdb-example
45. ShareJS http://sharejs.org/ | https://github.com/share/ShareJS RethinkDB http://rethinkdb.com/ | @rethinkdb livedb-rethinkdb https://github.com/thejsj/livedb-rethinkdb Twitter @thejsj Questions?