Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Concurrent Programming with Celluloid: JRubyConf

Concurrent Programming with Celluloid: JRubyConf

An introduction to Celluloid, a concurrent programming framework for Ruby based on the Actor Model. This is a version of the talk I gave at MWRC, but tailored for JRuby users, and with a much expanded intro to DCell.

tarcieri

May 23, 2012
Tweet

More Decks by tarcieri

Other Decks in Programming

Transcript

  1. Sidekiq What if 1 Sidekiq process could do the work

    of 20 Resque processes? http://mperham.github.com/sidekiq/
  2. ...but not quite •Designed from the ground up around Ruby

    and OOP •Conceptually harmonious and simple! •More advanced object model •Not quite as mature :|
  3. Less Boilerplate package typedactordemo import akka.actor._ import akka.dispatch.Future import akka.pattern.ask

    import akka.util.Timeout import akka.util.duration._ case class Request(payload: String) case class Response(payload: String) trait Service { def request(r: Request): Future[Response] } class ServiceImpl extends Service { val actor = { val ctx = TypedActor.context ctx.actorOf(Props[ServiceActor]) } implicit val timeout = Timeout(10 seconds) def request(req: Request): Future[Response] = (actor ? req).mapTo[Response] } class ServiceActor extends Actor { def receive = { case Request(payload) => sender ! Response(payload) } } object Main extends App { val system = ActorSystem("TypedActorDemo") val service: Service = TypedActor(system).typedActorOf( TypedProps[ServiceImpl]() ) val req = Request("hello world!") service.request(req) onSuccess { case Response(response) => println(response) system.shutdown() } }
  4. Actor Model •Actors are computational entities that can receive messages

    •Each actor has a unique address •If you know an actor’s address, you can send it messages •Actors can create new actors
  5. require 'thread' class ConcurrentNestedHash def initialize @outer = {} @mutex

    = Mutex.new end def [](*keys) @mutex.synchronize { keys.inject(@outer) { |h,k| h[k] } } end def []=(*args) @mutex.synchronize do value = args.pop raise ArgumentError, "wrong number of arguments (1 for 2)" if args.empty? key = args.pop hash = args.inject(@outer) { |h,k| h[k] ||= {} } hash[key] = value end end def inspect; @mutex.synchronize { super }; end end
  6. >> h = ConcurrentNestedHash.new => #<ConcurrentNestedHash:0x007f99ed735f08 @outer={}, @mutex=#<Mutex: 0x007f99ed735e90>> >>

    h[:foo, :bar, :baz] = 42 => 42 >> h => #<ConcurrentNestedHash:0x007f99ed735f08 @outer={:foo=>{:bar=>{:baz=>42}}}, @mutex=#<Mutex:0x007f99ed735e90>> >> h[:foo, :bar, :baz] => 42
  7. -require 'thread' +require 'celluloid' class ConcurrentNestedHash + include Celluloid def

    initialize @outer = {} - @mutex = Mutex.new end def [](*keys) - @mutex.synchronize { keys.inject(@outer) { |h,k| h[k] } } + keys.inject(@outer) { |h,k| h[k] } end def []=(*args) - @mutex.synchronize do value = args.pop raise ArgumentError, "wrong number of arguments (1 for 2)" if args.empty? key = args.pop hash = args.inject(@outer) { |h,k| h[k] ||= {} } hash[key] = value - end end - - def inspect; @mutex.synchronize { super }; end end
  8. require 'celluloid' class ConcurrentNestedHash include Celluloid def initialize @outer =

    {} end def [](*keys) keys.inject(@outer) { |h,k| h[k] } end def []=(*args) value = args.pop raise ArgumentError, "wrong number of arguments (1 for 2)" if args.empty? key = args.pop hash = args.inject(@outer) { |h,k| h[k] ||= {} } hash[key] = value end end
  9. “I thought of objects being like biological cells and/or individual

    computers on a network, only able to communicate with messages” - Alan Kay, creator of Smalltalk, on the meaning of "object oriented programming"
  10. =

  11. Concurrent Objects &DOOHU &HOOXORLG $FWRU3UR[\ &$// 5HFHLYHU &HOOXORLG 0DLOER[ 5(63216(

    &HOOXORLG 0DLOER[ &HOOXORLG$FWRU &HOOXORLG&DOO &HOOXORLG5HVSRQVH
  12. # Class methods added to classes which include Celluloid module

    ClassMethods # Create a new actor def new(*args, &block) proxy = Actor.new(allocate).proxy proxy._send_(:initialize, *args, &block) proxy end ...
  13. EXTREME Late Binding &DOOHU &HOOXORLG $FWRU3UR[\ &$// 5HFHLYHU &HOOXORLG 0DLOER[

    5(63216( &HOOXORLG 0DLOER[ &HOOXORLG$FWRU &HOOXORLG&DOO &HOOXORLG5HVSRQVH
  14. >> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}> >> h.inspect =>

    “#<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>” Synchronous Calls
  15. Futures &DOOHU &HOOXORLG $FWRU3UR[\ 5HFHLYHU &HOOXORLG 0DLOER[ &HOOXORLG$FWRU &HOOXORLG&DOO &$//

    &HOOXORLG 0DLOER[ )8785( &HOOXORLG )XWXUH &HOOXORLG5HVSRQVH 9$/8(" 9$/8(
  16. >> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}> >> future =

    h.future :inspect => #<Celluloid::Future:0x3ff3b953821a> >> 41 + 1 # roflscale computation => 42 >> future.value => “#<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>” Futures
  17. >> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}> >> future =

    h.future :inspect => #<Celluloid::Future:0x3ff3b953821a> >> 41 + 1 # roflscale computation => 42 >> future.value => “#<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>” Futures
  18. require 'celluloid' class Person include Celluloid def name self.class.to_s end

    def greet(interested_party) "Hello #{interested_party.name}, I'm #{name}" end end class Joe < Person; end class Mike < Person def greet(other) super << "\n" << other.greet(current_actor) end end mike = Mike.new joe = Joe.new puts mike.greet(joe)
  19. require 'celluloid' class Person include Celluloid def name self.class.to_s end

    def greet(interested_party) "Hello #{interested_party.name}, I'm #{name}" end end class Joe < Person; end class Mike < Person def greet(other) super << "\n" << other.greet(current_actor) end end mike = Mike.new joe = Joe.new puts mike.greet(joe)
  20. 0LNH -RH 0LNHJUHHW -RHJUHHW 0LNHQDPH  J U H H

    W  + H OO R  0 L N H    J U H H W  0 L N H  6XVSHQGHG Multitasking Fibers
  21. Call Benchmark Core i7 2.0GHz (OS X 10.7.3) 16815/s (60µs)

    - JRuby 1.6.7 20899/s (50µs) - JRuby HEAD
  22. Kilim Fibers •JRuby GSoC project by Miloš Hadžić •“Weave” the

    JRuby runtime, JIT •Fast, cheap Fibers on JRuby! •50% chance of success :|
  23. Fault Tolerance •Supervisors & Supervision Trees •“Fail early”, restart in

    a clean state •Do what Erlang does •No seriously, do what Erlang does
  24. Built on 0MQ •Fast, brokerless message queue •DCell uses push/pull

    sockets •Built on Celluloid::ZMQ/ffi-rzmq
  25. •Distributed process orchestration (e.g. Chef, Puppet) •Multi-tier Web Applications •Asynchronous

    background jobs Use Cases •Distributed process orchestration (e.g. Chef, Puppet) •Multi-tier Web Applications •Asynchronous background jobs
  26. Why do this? )URQW(QG )URQW(QG )URQW(QG 5(67 &OLHQW 5(67 &OLHQW

    5(67 &OLHQW 5(67 6HUYLFH 5(67 6HUYLFH 5(67 6HUYLFH 'RPDLQ 2EMHFW 'RPDLQ 2EMHFW 'RPDLQ 2EMHFW 'RPDLQ 2EMHFW 'RPDLQ 2EMHFW 'RPDLQ 2EMHFW 'RPDLQ 2EMHFW 'RPDLQ 2EMHFW 'RPDLQ 2EMHFW
  27. Instead of this? )URQW(QG )URQW(QG )URQW(QG 'RPDLQ 2EMHFW 'RPDLQ 2EMHFW

    'RPDLQ 2EMHFW 'RPDLQ 2EMHFW 'RPDLQ 2EMHFW 'RPDLQ 2EMHFW 'RPDLQ 2EMHFW 'RPDLQ 2EMHFW 'RPDLQ 2EMHFW
  28. "Objects can message objects transparently that live on other machines

    over the network, and you don't have to worry about the networking gunk, and you don't have to worry about finding them, and you don't have to worry about anything. It's just as if you messaged an object that's right next door." --Steve Jobs describing NeXT Portable Distributed Objects
  29. “itchy” commands >> require 'dcell' => true >> DCell.start :id

    => 'itchy', :addr => 'tcp:// 127.0.0.1:7777' I, [2012-05-09T10:20:46.999000 #52416] INFO -- : Connected to itchy => #<Celluloid::Supervisor(DCell::Group):0x836> Note itchy is on port 7777
  30. “scratchy” commands >> require 'dcell' => true >> DCell.start :id

    => 'scratchy', :addr => 'tcp:// 127.0.0.1:7778', :directory => {:id => 'itchy', :addr => 'tcp://127.0.0.1:7777'} I, [2012-05-09T10:26:42.322000 #52555] INFO -- : Connected to itchy I, [2012-05-09T10:26:42.331000 #52555] INFO -- : Connected to scratchy => #<Celluloid::Supervisor(DCell::Group):0x838> Note scratchy is on port 7778
  31. >> info_service = DCell::Node['itchy'][:info] => #<Celluloid::Actor(DCell::InfoService:0x83c) @platform="java" @ruby_platform="jruby 1.6.7" @cpu_type="Intel(R)

    Core(TM) i7-2635QM CPU" @ruby_version="1.9.2" @ruby_engine="jruby" @cpu_vendor=:intel @distribution="Mac OS X 10.7.3 (11D50b)" @cpu_count=8 @hostname="wintermute.local" @cpu_speed=2.0 @os="darwin" @cpu_arch="x86_64" @os_version="11.3.0"> >> info_service.uptime => 42 >> info_service.load_averages => [1.08, 0.93, 0.84] >> info_service.distribution => "Mac OS X 10.7.3 (11D50b)" “scratchy” commands
  32. Hello World Benchmark # httperf --num-conns=50 --num-calls=1000 Ruby Version Throughput

    Latency ------------ ---------- ------- JRuby HEAD 5650 reqs/s (0.2 ms/req) Ruby 1.9.3 5263 reqs/s (0.2 ms/req) JRuby 1.6.7 4303 reqs/s (0.2 ms/req) rbx HEAD 2288 reqs/s (0.4 ms/req)
  33. Hello World Comparison Web Server Throughput Latency ---------- ---------- -------

    Goliath (0.9.4) 2058 reqs/s (0.5 ms/req) Thin (1.2.11) 7502 reqs/s (0.1 ms/req) Node.js (0.6.5) 11735 reqs/s (0.1 ms/req) Ruby servers on 1.9.3
  34. Goals •Built out of parts of Rails •Webmachine for resources

    •Multithreaded development mode •Easy scatter/gather for SOA