Streaming APIs with Ruby

An introduction to RESTful streaming HTTP APIs and how to implement one with Rack::Stream

1. Streaming APIs with Ruby Jerry Cheung Monday, June 4, 12
2. Web 1.0 Monday, June 4, 12
3. Static Web • page is fixed after render • server does all work • one request, one page Monday, June 4, 12
4. Monday, June 4, 12
5. Web 2.0 Monday, June 4, 12
6. AJAX Web • break pages into “pagelets” • dynamic updates inline • one request, N-pagelets Monday, June 4, 12
7. Monday, June 4, 12
8. Many requests, but user stays on the same page Monday, June 4, 12
9. Web 3.0 Monday, June 4, 12
10. App Web • browser not the only client • rich UX, native-feel, desktop parity Monday, June 4, 12
11. Comet Web application model in which a long- held HTTP request allows a web server to push data to a browser, without the browser explicitly requesting it. Monday, June 4, 12
12. Monday, June 4, 12
13. Trend Monday, June 4, 12
14. server-side Web 1.0 Trend Monday, June 4, 12
15. server-side Web 1.0 JS server-side Web 2.0 Trend Monday, June 4, 12
16. server-side Web 1.0 JS server-side Web 2.0 API web, mobile, 3rd party Web 3.0 Trend Monday, June 4, 12
17. WebSockets var socket = new Websocket('ws://example.com/resource'); socket.onopen(function() { // connection established }); socket.onmessage(function(message) { console.log(message.data); }) socket.onclose(function() { // connection closing }); Monday, June 4, 12
18. WebSocket • + simple interface • + web standard • - not available everywhere Monday, June 4, 12
19. EventSource source = new EventSource('http://example.com/resource'); source.addEventListener('message', function(e) { console.log(e.data); }); Monday, June 4, 12
20. EventSource • + simple interface • + web standard • - not available everywhere • - unidirectional subset of WebSocket Monday, June 4, 12
21. AJAX Long Polling • + available almost everywhere • - browser keeps long running connections • - extra book-keeping for reconnect, messages Monday, June 4, 12
22. WebSocket Emulation • Firehose.io • socket.io • Pusher • Faye • flash sockets • more... Monday, June 4, 12
23. Rack::Stream • rackable - works with Sinatra, Rails, rack middleware • simple interface - #chunk(message) • multi-transport - WebSocket, EventSource, Long-poll Monday, June 4, 12
24. Rack-able require 'sinatra' use Rack::Stream get '/' do status 200 headers 'Content-Type' => 'text/plain' "Hello World" end Monday, June 4, 12
25. Rack-able require 'sinatra' use Rack::Stream get '/' do status 200 headers 'Content-Type' => 'text/plain' "Hello World" end ‘chunked’ Transfer-Encoding Monday, June 4, 12
26. Rack-able require 'sinatra' use Rack::Stream get '/' do status 200 headers 'Content-Type' => 'text/plain' "Hello World" end ‘chunked’ Transfer-Encoding Response bodies streamed out to clients Monday, June 4, 12
27. Multi-Transport Rack::Stream + Sinatra WebSocket EventSource HTTP Monday, June 4, 12
28. Stream to 100 require 'sinatra' use Rack::Stream # Starting... # 0 # 1 # ... # 99 get '/hundred' do 100.times.each do |i| chunk(i) end "Starting..." end Monday, June 4, 12
29. Stream to 100 require 'sinatra' use Rack::Stream # Starting... # 0 # 1 # ... # 99 get '/hundred' do 100.times.each do |i| chunk(i) end "Starting..." end stream response bodies Monday, June 4, 12
30. Stream to 100 require 'sinatra' use Rack::Stream # Starting... # 0 # 1 # ... # 99 get '/hundred' do 100.times.each do |i| chunk(i) end "Starting..." end enqueue content to stream stream response bodies Monday, June 4, 12
31. Persistent Connections • connection closes when there’s no data left • keep “chunking” data • don’t close the connection Monday, June 4, 12
32. Chat class API < Grape::API # snip... resources :messages do get do after_open do # subscribe after_open b/c this runs # until the connection is closed redis.subscribe 'messages' do |on| on.message do |channel, msg| chunk msg end end end # snip... end post do redis.publish 'messages', params[:text] end end end Monday, June 4, 12
33. Chat class API < Grape::API # snip... resources :messages do get do after_open do # subscribe after_open b/c this runs # until the connection is closed redis.subscribe 'messages' do |on| on.message do |channel, msg| chunk msg end end end # snip... end post do redis.publish 'messages', params[:text] end end end Define a resource Monday, June 4, 12
34. Chat class API < Grape::API # snip... resources :messages do get do after_open do # subscribe after_open b/c this runs # until the connection is closed redis.subscribe 'messages' do |on| on.message do |channel, msg| chunk msg end end end # snip... end post do redis.publish 'messages', params[:text] end end end Define a resource create messages Monday, June 4, 12
35. Chat class API < Grape::API # snip... resources :messages do get do after_open do # subscribe after_open b/c this runs # until the connection is closed redis.subscribe 'messages' do |on| on.message do |channel, msg| chunk msg end end end # snip... end post do redis.publish 'messages', params[:text] end end end Define a resource create messages stream new messages Monday, June 4, 12
36. Compose with Rack map '/' do run Chat::Application end map '/messages' do use Rack::Stream run API end Rack::Stream + Grape Rails Monday, June 4, 12
37. Manipulating Streams use Rack::Stream # outputs "HELLO WORLD" get '/' do before_chunk do |chunks| chunks.map(&:upcase) end chunk "world" "hello " end Monday, June 4, 12
38. Roadmap • Currently only works with Thin • Additional transport handlers • Load and compatibility testing Monday, June 4, 12
39. Thanks! https://github.com/intridea/rack-stream Tutorial: http://bit.ly/KkEC3O @whatcodecraves http://github.com/jch Monday, June 4, 12