A Mirage of Modules

A last minute talk to the Dagstuhl Algebraic Effects seminar crowd in April 2018, based on the St Andrews seminars.

1. A Mirage of Modules Anil Madhavapeddy,
 Dagstuhl Apr 2018 a last minute talk to the algebraic effects crowd
4. Hardware Unikernel TCP/IP HTTP libcamlrun OCaml code Xen hypervisor Xenstore
5. Hardware Unikernel TCP/IP HTTP libcamlrun OCaml code Almost all safe OCaml code in the result Xen hypervisor Xenstore Developer needed to know nothing about kernel programming ! From logic to the device drivers and network stack
6. MirageOS let hello () = Logs.info (fun fn -> fn "Hello World") Let’s build a hello world in OCaml, and then turn it into a unikernel progressively. First the basic fragment:
7. MirageOS let hello () = Logs.info (fun fn -> fn "Hello World") Let’s build a hello world in OCaml, and then turn it into a unikernel progressively. First the basic fragment: The Logs library does “lazy” logging Its argument is a higher order function fn is called with an argument to do logging
8. MirageOS Now make the program loop recursively. Needs a notion of time from the outside. let rec hello () = Logs.info (fun l -> l "Hello World"); Unix.sleep 1; hello ()
9. let rec hello () = Logs.info (fun l -> l "Hello World"); Unix.sleep 1; hello () MirageOS Now make the program loop recursively. Needs a notion of time from the outside. rec marks a recursive value The Unix call here pulls in the entire operating system Do we really need 15 millions lines of code to sleep for a second?
10. MirageOS open Lwt let rec hello () = Logs.info (fun l -> l "Hello World"); Lwt.bind (sleep 1.0) (fun () -> hello ()) let _ = Lwt_main.run (fun () -> hello ()) We implement concurrency in OCaml via the Lwt cooperative thread library.
11. MirageOS We implement concurrency in OCaml via the Lwt cooperative thread library. bind a promise to call next function after current thread finishes sleep builds a priority queue of waiting threads when 1s has passed, hello is called open Lwt let rec hello () = Logs.info (fun l -> l "Hello World"); Lwt.bind (sleep 1.0) (fun () -> hello ()) let _ = Lwt_main.run (fun () -> hello ())
12. MirageOS We implement concurrency in OCaml via the Lwt cooperative thread library. open Lwt let rec hello () = Logs.info (fun l -> l "Hello World"); Lwt.bind (sleep 1.0) (fun () -> hello ()) let _ = Lwt_main.run (fun () -> hello ())
13. MirageOS We implement concurrency in OCaml via the Lwt cooperative thread library. open Lwt.Infix let rec hello () = Logs.info (fun l -> l "Hello World"); sleep 1.0 >>= fun () -> hello () let _ = Lwt_main.run (fun () -> hello ())
14. MirageOS We implement concurrency in OCaml via the Lwt cooperative thread library. operators make the cooperative threading more natural to write open Lwt.Infix let rec hello () = Logs.info (fun l -> l "Hello World"); sleep 1.0 >>= fun () -> hello () let _ = Lwt_main.run (fun () -> hello ()) but the types all change :(
 if only we could handle the effects more naturally
15. MirageOS Now let’s implement this as a unikernel! open Lwt.Infix module Hello(Time : TIME) = struct let start _ = let rec loop () = Logs.info (fun f -> f "hello"); Time.sleep_ns (Duration.of_sec 1) >>= fun () -> loop () in loop () end
16. MirageOS Now let’s implement this as a unikernel! abstract code in a module that takes a Time signature as a parameter No assumption of which Time implementation open Lwt.Infix module Hello(Time : TIME) = struct let start _ = let rec loop () = Logs.info (fun f -> f "hello"); Time.sleep_ns (Duration.of_sec 1) >>= fun () -> loop () in loop () end
17. MirageOS The TIME signature is very abstract module Hello(Time : TIME) = struct let start _ = let rec loop () = Logs.info (fun f -> f "hello"); Time.sleep_ns (Duration.of_sec 1) >>= fun () -> loop () in loop () end module type TIME = sig type +'a io (** The type for a potentially blocking I/O operation *) val sleep_ns: int64 -> unit io (** [sleep_ns n] Block the current thread for [n] nanoseconds *) end
18. MirageOS The TIME signature is very abstract abstract code in a module that takes a Time signature as a parameter We have many implementations of TIME in different hardware contexts module type TIME = sig type +'a io (** The type for a potentially blocking I/O operation *) val sleep_ns: int64 -> unit io (** [sleep_ns n] Block the current thread for [n] nanoseconds *) end Depend on “just” the operating system functionality needed
19. MirageOS To compile this code, we supply the Mirage CLI with the hardware target, and it links the right OCaml libraries for the job for that platform. $ mirage configure -t unix hello.ml $ make # builds a Unix binary $ mirage configure -t xen hello.ml $ make # builds an entire Xen unikernel for development for deployment
20. •We can’t manually assemble all these combinations for every device in the world. •There are so many resource and policies: •a filesystem requires a block device •an encryption layer requires strong entropy •do two tcp/ip stacks share the same ethernet? •Can we express these configurations in a more principled fashion? But how do we program these in practise?
21. • Functoria is an OCaml domain-specific language that: • describes modules and signatures • ascribes types to groups of them • can generate code to produce an executable application. Functoria
22. • Mirage is so-named because it disappears behind a set of signatures that describe all resources. • Applications consist of a series of small, composition modules that are abstracted over the resources they need. Functoria
23. let main = foreign “Unikernel.Main” (console @-> time @-> job) let () = register “console" [main $ default_console $ default_time ]
24. let main = foreign “Unikernel.Main” (console @-> time @-> job) let () = register “console" [main $ default_console $ default_time ] module Main (Console: Mirage_types_lwt.CONSOLE) (Time: Mirage_types_lwt.TIME) = struct let start c _ = let rec loop = function | 0 -> Lwt.return_unit | n -> Console.log c "hello" >>= fun () -> Time.sleep_ns (Duration.of_sec 1) >>= fun () -> Console.log c "world" >>= fun () -> loop (pred n) in loop 4 end
25. let main = foreign “Unikernel.Main” (console @-> time @-> job) let () = register “console" [main $ default_console $ default_time ] $ mirage describe -t unix —dot
26. let main = foreign “Unikernel.Main” (console @-> time @-> job) let () = register “console" [main $ default_console $ default_time ] $ mirage describe -t xen —dot
27. let main = foreign “Unikernel.Main” (console @-> time @-> job) let () = register “console" [main $ default_console $ default_time ] (* Generated by mirage configure -t xen *) <snip> module Unikernel1 = Unikernel.Main(Console_xen)(OS.Time) module Mirage_logs1 = Mirage_logs.Make(Pclock) <snip> let () = let t = Lazy.force noop1 >>= fun _ -> Lazy.force noop1 >>= fun _ -> Lazy.force key1 >>= fun _ -> Lazy.force mirage_logs1 >>= fun _ -> Lazy.force mirage1 in run t
28. Configuration of unikernel is staged by the mirage tool, to generate a command-line interface specialised to it. Functoria: keys The keys can be supplied at build time, or dynamically at execution time. The more that is supplied at build time, the better the application gets!
29. let disk = generic_kv_ro "t" let main = foreign "Unikernel.Main" (kv_ro @-> job) let () = register "kv_ro" [main $ disk]
30. let disk = generic_kv_ro "t" let main = foreign "Unikernel.Main" (kv_ro @-> job) let () = register "kv_ro" [main $ disk] Ways to build to implement a key/value store: • Crunch the files directly into the binary. No external devices needed, but all the files need to be in memory. • Passthrough to an underlying Unix filesystem. • Construct a key/value device from an arbitrary filesystem implementation. • Tar an archive from an underlying block device.
31. let disk = generic_kv_ro "t" let main = foreign "Unikernel.Main" (kv_ro @-> job) let () = register "kv_ro" [main $ disk] $ mirage describe -t unix —dot
32. let disk = generic_kv_ro "t" let main = foreign "Unikernel.Main" (kv_ro @-> job) let () = register "kv_ro" [main $ disk] $ mirage describe -t unix —dot Crunch Passthrough Mmap Tar
33. let disk = generic_kv_ro "t" let main = foreign "Unikernel.Main" (kv_ro @-> job) let () = register "kv_ro" [main $ disk] Crunch Passthrough Mmap Tar $ mirage describe -t unix —kv_ro=fat —dot
34. let disk = generic_kv_ro "t" let main = foreign "Unikernel.Main" (kv_ro @-> job) let () = register "kv_ro" [main $ disk] $ mirage describe -t unix —kv_ro=fat —dot Mmap
35. let disk = generic_kv_ro "t" let main = foreign "Unikernel.Main" (kv_ro @-> job) let () = register "kv_ro" [main $ disk] $ mirage describe -t xen —dot
36. let disk = generic_kv_ro "t" let main = foreign "Unikernel.Main" (kv_ro @-> job) let () = register "kv_ro" [main $ disk] $ mirage describe -t xen —dot Crunch Tar Mmap
37. let disk = generic_kv_ro "t" let main = foreign "Unikernel.Main" (kv_ro @-> job) let () = register "kv_ro" [main $ disk] $ mirage describe -t xen —dot Crunch Tar Mmap No option for a “passthrough” filesystem in Xen mode as there is no Unix operating system available!
38. let disk = generic_kv_ro "t" let main = foreign "Unikernel.Main" (kv_ro @-> job) let () = register "kv_ro" [main $ disk] $ mirage describe —dot
39. let disk = generic_kv_ro "t" let main = foreign "Unikernel.Main" (kv_ro @-> job) let () = register "kv_ro" [main $ disk] $ mirage describe —dot
40. let disk = generic_kv_ro "t" let main = foreign "Unikernel.Main" (kv_ro @-> job) let () = register "kv_ro" [main $ disk] $ mirage describe —dot Some options must be specified at build time (e.g. the hardware target) Others can remain dynamic if desired, at the expense of less specialisation
41. let handler = foreign "Unikernel.Main" (conduit @-> job) let () = register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] Let’s consider an example with networking. This can be even more complicated, with many ways to configure all the various pieces of the stack. Functoria absolutely shines here! Let’s look at a static web server…
42. let handler = foreign "Unikernel.Main" (conduit @-> job) let () = register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] module Main (CON:Conduit_mirage.S) = struct let src = Logs.Src.create "conduit_server" ~doc:"Conduit HTTP server" module Log = (val Logs.src_log src: Logs.LOG) module H = Cohttp_mirage.Server(Conduit_mirage.Flow) let start conduit = let http_callback _conn_id req _body = let path = Uri.path (Cohttp.Request.uri req) in Log.debug (fun f -> f "Got request for %s\n" path); H.respond_string ~status:`OK ~body:"hello mirage world!\n" () in let spec = H.make ~callback:http_callback () in CON.listen conduit (`TCP 80) (H.listen spec) end
43. let handler = foreign "Unikernel.Main" (conduit @-> job) let () = register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] $ mirage describe -t unix --net=socket --dhcp=false --dot
44. let handler = foreign "Unikernel.Main" (conduit @-> job) let () = register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] $ mirage describe -t unix --net=socket --dhcp=false --dot
45. let handler = foreign "Unikernel.Main" (conduit @-> job) let () = register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] $ mirage describe -t unix --net=socket --dhcp=false --dot This binary hooks into the Unix socket stack and is just like a normal application
46. let handler = foreign "Unikernel.Main" (conduit @-> job) let () = register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] $ mirage describe -t xen --dot In Xen, we need to hook up an entire networking stack from scratch, since there is no OS support
47. let handler = foreign "Unikernel.Main" (conduit @-> job) let () = register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] $ mirage describe -t xen --dot In Xen, we need to hook up an entire networking stack from scratch, since there is no OS support No worries! We have written an entire TCP/IP stack in this functor style
48. let handler = foreign "Unikernel.Main" (conduit @-> job) let () = register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] $ mirage describe -t xen --dot
49. let handler = foreign "Unikernel.Main" (conduit @-> job) let () = register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] $ mirage describe -t xen --dot
50. let handler = foreign "Unikernel.Main" (conduit @-> job) let () = register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] $ mirage describe -t xen --net=direct —dhcp=false --dot
51. let handler = foreign "Unikernel.Main" (conduit @-> job) let () = register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] $ mirage describe -t xen --net=direct —dhcp=false --dot module Ethif1 = Ethif.Make(Netif) module Arpv41 = Arpv4.Make(Ethif1)(Mclock)(OS.Time) module Static_ipv41 = Static_ipv4.Make(Ethif1)(Arpv41) module Icmpv41 = Icmpv4.Make(Static_ipv41) module Udp1 = Udp.Make(Static_ipv41)(Stdlibrandom) module Tcp1 = Tcp.Flow.Make(Static_ipv41)(OS.Time) (Mclock)(Stdlibrandom) module Tcpip_stack_direct1 = Tcpip_stack_direct.Make(OS.Time) (Stdlibrandom)(Netif)(Ethif1)(Arpv41)(Static_ipv41) (Icmpv41)(Udp1)(Tcp1) module Conduit_mirage1 = Conduit_mirage.With_tcp(Tcpip_stack_direct1) module Unikernel1 = Unikernel.Main(Conduit_mirage) module Mirage_logs1 = Mirage_logs.Make(Pclock)
52. let handler = foreign "Unikernel.Main" (conduit @-> job) let () = register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] $ mirage describe -t xen --net=direct —dhcp=false --dot Name conduit_server Keys dhcp=false, interface=0 (default), ipv4=10.0.0.2/24 (default), ipv4-gateway=10.0.0.1 (default), logs= (default), net=direct, prng=stdlib (default), target=xen, target_debug=false, warn_error=false (default)
53. • Expresses complex system dependencies as a graph. • Models the “shapes” of the nodes using types, so that a network stack cannot accidentally become a storage device. • Incrementally specialises the graph as more information becomes available. • Generates code to turn a configuration set into a full unikernel. Functoria: recap
54. • The range of devices is now remarkable in MirageOS. The signatures are easily extensible as you just create a new one for a specific domain. • This is how we break the operating cycle of bloat. • Design modular interfaces that are fit for a purpose and no more. Functoria
55. • How far can functoria devices go? • We are building new “device drivers” for: • Remote vs local communication • Security key management • And even data science algorithms! Functoria
56. • Functors for abstraction to wire the different components • Effects for sequential but concurrent code - no more sort-of-monads • Staging to eliminate configurations and do build time specialisation. Key Things We Need to Replace All the Bad OS Things ✓ ✓ ?