Enable high coucurrent & availability with Goliath

Transcript

1. Enable high concurrent & availability with Goliath Von Stark Saturday, December 8, 12
2. Who I use Ruby, Neo4j, Riak co-founder of nosql.org.tw co-founder at NeoArk @vonstark32 http://vonstark.co http://neo4j.tw http://nosql.org.tw Saturday, December 8, 12
3. Background Ruby Mainly iPhone & Android Clients Requests : data analyze, data process, graph traversing ... Heavy I/O requests Saturday, December 8, 12
4. What we use before? Rails + Thin / Rainbows / Unicorn .. Standalone Worker ( background job, beanstalk, whenever, resque.... ) Padrino Pure EventMachine Saturday, December 8, 12
5. Concurrent is a Myth in Ruby Saturday, December 8, 12
6. Myths? Global interpreter lock Deciphering global interpreter lock process para$elism reactor pattern Saturday, December 8, 12
7. Server architectures & implementation Architectures Server Threaded Mongrel, jRuby (torqueBox) Fork Passenger, Unicorn Evented Thin, Goliath Implementation GIL Fiber Thread MRI 1.8 Y N Green MRI 1.9 Y Y OS jRuby N Slow Java Threads Rubinius 1.x Y N Green, OS Rubinius 2.0 N Y OS Saturday, December 8, 12
8. So, how we concurrent in Ruby? Saturday, December 8, 12
9. EventMachine EventMachine provides a fast, lightweight %amework for implementing Ruby programs that can use the network to communicate with other processes. It does provide an alternate technique for those applications requiring better performance, scalability, and discipline over the behavior of network sockets, than is easily obtainable using the built-in libraries, especia$y in applications which are structura$y we$-suited for the event-driven programming model. Saturday, December 8, 12
10. Event Machine Components Deferrable Make object concurrent Spawned Process Erlang style processes Channel Pub/Sub Queue Queue and Pop Saturday, December 8, 12
11. It is powerful, but...... Saturday, December 8, 12
12. Callback Hell r1.callback{ r2.callback{ r3.callback{ r4.callback{ ..... } } } } Saturday, December 8, 12
13. Hybrid Code def http_get(url) f = Fiber.current http = EventMachine::HttpRequest.new(url).get # resume fiber once http call is done http.callback { f.resume(http) } http.errback { f.resume(http) } return Fiber.yield end EventMachine.run do Fiber.new{ page = http_get('http://www.google.com/') puts "Fetched page: #{page.response_header.status}" if page page = http_get('http://www.google.com/search?q=eventmachine') puts "Fetched page 2: #{page.response_header.status}" end }.resume end Saturday, December 8, 12
14. Hey, Ruby is not functional language.. Saturday, December 8, 12
15. Just grab Node.js , Erlang , Scala , Go , Haske$ ... Saturday, December 8, 12
16. Reasons Get advantages %om gems Easy to integrate with rails, sinatra... etc Ruby is Beautiful Ruby is Elegant Ruby is Agile Ruby is able to concurrent Saturday, December 8, 12
17. Good life, don’t you want? Saturday, December 8, 12
18. Goliath Framework High performance and concurrent Based on Event-Machine Event-Driven Been Proven ( Pusher.com, PostRank.com ) Easy to use Low Memory Footprint 2011.03 released Saturday, December 8, 12
19. Design Leverage fiber(coroutines) Simple configuration Middleware support fu$ asynchronous processing Readable & Maintainable code Saturday, December 8, 12
20. Performance MRI 1.9.2p136 3000 req/s Saturday, December 8, 12
21. require ‘goliath’ Saturday, December 8, 12
22. Routes with Goliath class Index < Goliath::API use Goliath::Rack::Params get '/', Index post '/login', Login get '/overview', Overview not_found() do run Proc.new{ [200, {'Content-Type' => 'text/plain'}, "Hi. Welcome to API. "] } end end Saturday, December 8, 12
23. Plugins module Goliath module Plugin class Log def initialize do_something_here... end def run do_something_here... end end end end class YourProject < Goliath::API use Goliath::Plugin::Log end Saturday, December 8, 12
24. Serve Static Files class YouProject < Goliath::API use(Rack::Static, :root => Goliath::Application.app_path("public"), :urls => ["/favicon.ico", '/stylesheets', '/javascripts', '/images']) end Saturday, December 8, 12
25. Session & Cookies class Auth < Goliath::API use Rack::Session::Cookie, :secret => 'change me' use Warden::Manager do |manager| manager.default_strategies :password manager.failure_app = Proc.new { |env| [401, {}, 'Access Denied'] } end use Authenticator end Saturday, December 8, 12
26. Use EM class Index< Goliath::API def response EM.defer{ long_long_execution() } end end class Channel< Goliath::API def response EM.channel.start do end end end class Timer< Goliath::API def response EventMachine::PeriodicTimer(10){ loop_work() } end end class Queue< Goliath::API def response queue = EventMachine::Queue.new end end Saturday, December 8, 12
27. Connect with databases? Data Mapper Sequel ActiveRecord Swi& or anything you like.... Saturday, December 8, 12
28. But it is not enough..... Saturday, December 8, 12
29. Straight forward methods class Index < Goliath::API def response(env) render :erb end end class Index < Goliath::API def response(env) @res={:status=>"success"} render :json end end Saturday, December 8, 12
30. Render with erb def response @hello="Hello World" render :erb end <%content_for :javascripts%> <%=javascript_include_tag “jquery”%> <%end%> Everybody. <%=@hello%> <%=partial ”partial/something.erb”%> Saturday, December 8, 12
31. MVC & Generator $> gm new Balalalala Saturday, December 8, 12
32. Rake Commands rake db:create rake db:rollback rake db:migrate rake db:drop rake db:redo rake db:console rake cluster:start rake cluster:reboot rake cluster:shutdown Saturday, December 8, 12
33. Key to Concurrent Make your each work as sma$ as possible Don’t block main loop (with em) Test performance for your each request Saturday, December 8, 12
34. Some experiences Use async adapter with your database ex : mysql2 Use better performance gems : sequel, erubis Database is next bottleneck, have one Nosql for your unimportant data ( feeds, likes ) Use Proc & Block with EM.defer or Thread Use em-http with your Network request Saturday, December 8, 12
35. Thanks Saturday, December 8, 12