Concurrent & Multicore OCaml: A deep dive

Talk was given at Facebook, Menlo Park.

1. Concurrent & Multicore OCaml: A deep dive KC Sivaramakrishnan1 & Stephen Dolan1 Leo White2, Jeremy Yallop1,3, Armaël Guéneau4, Anil Madhavapeddy1,3 1 2 3 4
2. Concurrency ≠ Parallelism • Concurrency • Programming technique • Overlapped execution of processes • Parallelism • (Extreme) Performance hack • Simultaneous execution of computations
3. Concurrency ≠ Parallelism • Concurrency • Programming technique • Overlapped execution of processes • Parallelism • (Extreme) Performance hack • Simultaneous execution of computations Concurrency ∩ Parallelism ➔ Scalable Concurrency
4. Concurrency ≠ Parallelism • Concurrency • Programming technique • Overlapped execution of processes • Parallelism • (Extreme) Performance hack • Simultaneous execution of computations Concurrency ∩ Parallelism ➔ Scalable Concurrency (Fibers) (Domains)
5. Schedulers • Multiplexing fibers over domain(s) • Bake scheduler into the runtime system (GHC)
6. Schedulers • Multiplexing fibers over domain(s) • Bake scheduler into the runtime system (GHC) • Allow programmers to describe schedulers! • Parallel search —> LIFO work-stealing • Web-server —> FIFO runqueue • Data parallel —> Gang scheduling
7. Schedulers • Multiplexing fibers over domain(s) • Bake scheduler into the runtime system (GHC) • Allow programmers to describe schedulers! • Parallel search —> LIFO work-stealing • Web-server —> FIFO runqueue • Data parallel —> Gang scheduling • Algebraic Effects and Handlers
8. Algebraic effects & handlers
9. • Programming and reasoning about computational effects in a pure setting. • Cf. Monads Algebraic effects & handlers
10. • Programming and reasoning about computational effects in a pure setting. • Cf. Monads • Eff — http://www.eff-lang.org/ Algebraic effects & handlers
11. Algebraic Effects: Example exception Foo of int let f () = 1 + (raise (Foo 3)) let r = try f () with Foo i -> i + 1
12. Algebraic Effects: Example exception Foo of int let f () = 1 + (raise (Foo 3)) let r = try f () with Foo i -> i + 1
13. Algebraic Effects: Example exception Foo of int let f () = 1 + (raise (Foo 3)) let r = try f () with Foo i -> i + 1 val r : int = 4
14. Algebraic Effects: Example exception Foo of int let f () = 1 + (raise (Foo 3)) let r = try f () with Foo i -> i + 1 val r : int = 4 effect Foo : int -> int let f () = 1 + (perform (Foo 3)) let r = try f () with effect (Foo i) k -> continue k (i + 1)
15. Algebraic Effects: Example exception Foo of int let f () = 1 + (raise (Foo 3)) let r = try f () with Foo i -> i + 1 val r : int = 4 effect Foo : int -> int let f () = 1 + (perform (Foo 3)) let r = try f () with effect (Foo i) k -> continue k (i + 1)
16. Algebraic Effects: Example exception Foo of int let f () = 1 + (raise (Foo 3)) let r = try f () with Foo i -> i + 1 val r : int = 4 effect Foo : int -> int let f () = 1 + (perform (Foo 3)) let r = try f () with effect (Foo i) k -> continue k (i + 1)
17. Algebraic Effects: Example exception Foo of int let f () = 1 + (raise (Foo 3)) let r = try f () with Foo i -> i + 1 val r : int = 4 effect Foo : int -> int let f () = 1 + (perform (Foo 3)) 4 let r = try f () with effect (Foo i) k -> continue k (i + 1)
18. Algebraic Effects: Example exception Foo of int let f () = 1 + (raise (Foo 3)) let r = try f () with Foo i -> i + 1 val r : int = 4 effect Foo : int -> int let f () = 1 + (perform (Foo 3)) 4 let r = try f () with effect (Foo i) k -> continue k (i + 1) val r : int = 5
19. Algebraic Effects: Example exception Foo of int let f () = 1 + (raise (Foo 3)) let r = try f () with Foo i -> i + 1 val r : int = 4 effect Foo : int -> int let f () = 1 + (perform (Foo 3)) 4 let r = try f () with effect (Foo i) k -> continue k (i + 1) val r : int = 5 fiber — lightweight stack
20. Scheduler Demo1 [1] https://github.com/kayceesrk/ocaml15-eff/tree/master/chameneos-redux
21. • Fibers: Heap allocated, dynamically resized stacks • ~10s of bytes • No unnecessary closure allocation costs unlike CPS Implementation
22. • Fibers: Heap allocated, dynamically resized stacks • ~10s of bytes • No unnecessary closure allocation costs unlike CPS • One-shot delimited continuations • Simplifies reasoning about resources - sockets, locks, etc. Implementation
23. • Fibers: Heap allocated, dynamically resized stacks • ~10s of bytes • No unnecessary closure allocation costs unlike CPS • One-shot delimited continuations • Simplifies reasoning about resources - sockets, locks, etc. • Handlers —> Linked-list of fibers Implementation
24. • Fibers: Heap allocated, dynamically resized stacks • ~10s of bytes • No unnecessary closure allocation costs unlike CPS • One-shot delimited continuations • Simplifies reasoning about resources - sockets, locks, etc. • Handlers —> Linked-list of fibers Implementation handle / continue handler sp call chain reference
25. Implementation handle / continue handle / continue sp handler call chain reference • Fibers: Heap allocated, dynamically resized stacks • ~10s of bytes • No unnecessary closure allocation costs unlike CPS • One-shot delimited continuations • Simplifies reasoning about resources - sockets, locks, etc. • Handlers —> Linked-list of fibers
26. perform sp handle / continue Implementation handler call chain reference • Fibers: Heap allocated, dynamically resized stacks • ~10s of bytes • No unnecessary closure allocation costs unlike CPS • One-shot delimited continuations • Simplifies reasoning about resources - sockets, locks, etc. • Handlers —> Linked-list of fibers
27. Native-code fibers — Vanilla C
28. Native-code fibers — Vanilla OCaml start program C OCaml
29. Native-code fibers — Vanilla OCaml start program C call C OCaml C
30. Native-code fibers — Vanilla OCaml start program C call OCaml callback C OCaml C OCaml
31. Native-code fibers — Vanilla OCaml start program C call OCaml callback C call C OCaml C OCaml C
32. Native-code fibers — Vanilla OCaml start program C call OCaml callback C call OCaml callback C OCaml C OCaml C OCaml
33. C system stack Native-code fibers — Effects
34. C system stack Native-code fibers — Effects OCaml heap OCaml start program
35. C system stack Native-code fibers — Effects OCaml heap OCaml start program handle
36. C system stack Native-code fibers — Effects OCaml heap OCaml start program C call handle C
37. C system stack Native-code fibers — Effects OCaml heap OCaml start program C call handle OCaml callback C
38. C system stack Native-code fibers — Effects OCaml heap OCaml start program C call handle OCaml callback C call C C
39. C system stack Native-code fibers — Effects OCaml heap OCaml start program C call handle OCaml callback C call C C 1. Stack overflow checks for OCaml functions • Simple static analysis eliminates many checks
40. C system stack Native-code fibers — Effects OCaml heap OCaml start program C call handle OCaml callback C call C C 1. Stack overflow checks for OCaml functions • Simple static analysis eliminates many checks 2. FFI calls are more expensive due to stack switching • Specialise for calls which {allocate / pass arguments on stack / do neither}
41. Performance : Vanilla OCaml 0 0.25 0.5 0.75 1 almabench alt-ergo-parameter_smallest_divisor alt-ergo-carte_autorisee_3 alt-ergo-parameter_relabel alt-ergo-OBF__ggjj_2 alt-ergo-parameter_def alt-ergo-parameter_def alt-ergo-OBF__yyll_1 alt-ergo-bbvv_351 alt-ergo-controler_carte_13 alt-ergo-div2_sub alt-ergo-parameter_def alt-ergo-ccgg_2055 alt-ergo-parameter_def alt-ergo-parameter_inverse_in_place alt-ergo-ccgg_1759 alt-ergo-parameter_def alt-ergo-induction_step alt-ergo-ccgg_1618 alt-ergo-ccgg_219 alt-ergo-advance_automaton_25 alt-ergo-nsec_sum_higher_than_1s alt-ergo-fill_assert_39_Alt-Ergo bdd chameneos-async chameneos-lwt cohttp-lwt core_micro cpdf-merge cpdf-reformat cpdf-squeeze cpdf-transform frama-c-deflate frama-c-idct js_of_ocaml jsontrip-sample kb kb-no-exc lexifi-g2pp menhir-fancy menhir-sql menhir-standard minilight numal-durand-kerner-aberth numal-fft numal-k-means numal-levinson-durbin numal-lu-decomposition numal-naive-multilayer numal-qr-decomposition numal-rnd_access numal-simple_access patdiff sauvola-contrast sequence sequence-cps setrip setrip-smallbuf thread-ring-async-pipe thread-ring-lwt-mvar thread-ring-lwt-stream thread-sleep-async thread-sleep-lwt valet-async valet-lwt ydump-sample 4.02.2+effects 4.02.2+vanilla Normalised time (lower is better)
42. Performance : Vanilla OCaml 0 0.25 0.5 0.75 1 almabench alt-ergo-parameter_smallest_divisor alt-ergo-carte_autorisee_3 alt-ergo-parameter_relabel alt-ergo-OBF__ggjj_2 alt-ergo-parameter_def alt-ergo-parameter_def alt-ergo-OBF__yyll_1 alt-ergo-bbvv_351 alt-ergo-controler_carte_13 alt-ergo-div2_sub alt-ergo-parameter_def alt-ergo-ccgg_2055 alt-ergo-parameter_def alt-ergo-parameter_inverse_in_place alt-ergo-ccgg_1759 alt-ergo-parameter_def alt-ergo-induction_step alt-ergo-ccgg_1618 alt-ergo-ccgg_219 alt-ergo-advance_automaton_25 alt-ergo-nsec_sum_higher_than_1s alt-ergo-fill_assert_39_Alt-Ergo bdd chameneos-async chameneos-lwt cohttp-lwt core_micro cpdf-merge cpdf-reformat cpdf-squeeze cpdf-transform frama-c-deflate frama-c-idct js_of_ocaml jsontrip-sample kb kb-no-exc lexifi-g2pp menhir-fancy menhir-sql menhir-standard minilight numal-durand-kerner-aberth numal-fft numal-k-means numal-levinson-durbin numal-lu-decomposition numal-naive-multilayer numal-qr-decomposition numal-rnd_access numal-simple_access patdiff sauvola-contrast sequence sequence-cps setrip setrip-smallbuf thread-ring-async-pipe thread-ring-lwt-mvar thread-ring-lwt-stream thread-sleep-async thread-sleep-lwt valet-async valet-lwt ydump-sample 4.02.2+effects 4.02.2+vanilla Normalised time (lower is better) 4.02.2+effects ~5.4% slower
43. Performance : Chameneos-Redux Time (S) 0 0.45 0.9 1.35 1.8 Iterations (X100,000) 1 2 3 4 5 6 7 8 9 10 Lwt Concurrency Monad GHC Fibers
44. Generator from Iterator1 [1] https://github.com/kayceesrk/ocaml15-eff/blob/master/generator.ml let rec iter f = function | Leaf -> () | Node (l, x, r) -> iter f l; f x; iter f r type 'a t = | Leaf | Node of 'a t * 'a * 'a t
45. Generator from Iterator1 [1] https://github.com/kayceesrk/ocaml15-eff/blob/master/generator.ml (* val to_gen : 'a t -> (unit -> 'a option) *) let to_gen (type a) (t : a t) = let module M = struct effect Next : a -> unit end in let open M in let step = ref (fun () -> assert false) in let first_step () = try iter (fun x -> perform (Next x)) t; None with effect (Next v) k -> step := continue k; Some v in step := first_step; fun () -> !step () let rec iter f = function | Leaf -> () | Node (l, x, r) -> iter f l; f x; iter f r type 'a t = | Leaf | Node of 'a t * 'a * 'a t
46. Performance : Generator Time (S) 0 1 2 3 4 Binary tree depth 15 16 17 18 19 20 21 22 23 24 25 Iterator Fiber Generator H/W Generator
47. Async I/O in direct style1 [1] https://github.com/kayceesrk/ocaml15-eff/tree/master/async-io
48. Async I/O in direct style1 Callback Hell [1] https://github.com/kayceesrk/ocaml15-eff/tree/master/async-io
49. Javascript backend • js_of_ocaml • OCaml bytecode —> Javascript
50. Javascript backend • js_of_ocaml • OCaml bytecode —> Javascript • js_of_ocaml compiler pass • Whole-program selective CPS transformation
51. Javascript backend • js_of_ocaml • OCaml bytecode —> Javascript • js_of_ocaml compiler pass • Whole-program selective CPS transformation • Work-in-progress! • Runs “hello-effects-world”!
52. fin. https://github.com/kayceesrk/ocaml-eff-example