OCaml code in the result Xen hypervisor Xenstore Developer needed to know nothing about kernel programming ! From logic to the device drivers and network stack
"Hello World") Let’s build a hello world in OCaml, and then turn it into a unikernel progressively. First the basic fragment: The Logs library does “lazy” logging Its argument is a higher order function fn is called with an argument to do logging
"Hello World"); Unix.sleep 1; hello () MirageOS Now make the program loop recursively. Needs a notion of time from the outside. rec marks a recursive value The Unix call here pulls in the entire operating system Do we really need 15 millions lines of code to sleep for a second?
l -> l "Hello World"); Lwt.bind (sleep 1.0) (fun () -> hello ()) let _ = Lwt_main.run (fun () -> hello ()) We implement concurrency in OCaml via the Lwt cooperative thread library.
thread library. bind a promise to call next function after current thread finishes sleep builds a priority queue of waiting threads when 1s has passed, hello is called open Lwt let rec hello () = Logs.info (fun l -> l "Hello World"); Lwt.bind (sleep 1.0) (fun () -> hello ()) let _ = Lwt_main.run (fun () -> hello ())
thread library. operators make the cooperative threading more natural to write open Lwt.Infix let rec hello () = Logs.info (fun l -> l "Hello World"); sleep 1.0 >>= fun () -> hello () let _ = Lwt_main.run (fun () -> hello ()) but the types all change :( if only we could handle the effects more naturally
module Hello(Time : TIME) = struct let start _ = let rec loop () = Logs.info (fun f -> f "hello"); Time.sleep_ns (Duration.of_sec 1) >>= fun () -> loop () in loop () end
in a module that takes a Time signature as a parameter No assumption of which Time implementation open Lwt.Infix module Hello(Time : TIME) = struct let start _ = let rec loop () = Logs.info (fun f -> f "hello"); Time.sleep_ns (Duration.of_sec 1) >>= fun () -> loop () in loop () end
TIME) = struct let start _ = let rec loop () = Logs.info (fun f -> f "hello"); Time.sleep_ns (Duration.of_sec 1) >>= fun () -> loop () in loop () end module type TIME = sig type +'a io (** The type for a potentially blocking I/O operation *) val sleep_ns: int64 -> unit io (** [sleep_ns n] Block the current thread for [n] nanoseconds *) end
a module that takes a Time signature as a parameter We have many implementations of TIME in different hardware contexts module type TIME = sig type +'a io (** The type for a potentially blocking I/O operation *) val sleep_ns: int64 -> unit io (** [sleep_ns n] Block the current thread for [n] nanoseconds *) end Depend on “just” the operating system functionality needed
with the hardware target, and it links the right OCaml libraries for the job for that platform. $ mirage configure -t unix hello.ml $ make # builds a Unix binary $ mirage configure -t xen hello.ml $ make # builds an entire Xen unikernel for development for deployment
in the world. •There are so many resource and policies: •a filesystem requires a block device •an encryption layer requires strong entropy •do two tcp/ip stacks share the same ethernet? •Can we express these configurations in a more principled fashion? But how do we program these in practise?
of signatures that describe all resources. • Applications consist of a series of small, composition modules that are abstracted over the resources they need. Functoria
let () = register “console" [main $ default_console $ default_time ] module Main (Console: Mirage_types_lwt.CONSOLE) (Time: Mirage_types_lwt.TIME) = struct let start c _ = let rec loop = function | 0 -> Lwt.return_unit | n -> Console.log c "hello" >>= fun () -> Time.sleep_ns (Duration.of_sec 1) >>= fun () -> Console.log c "world" >>= fun () -> loop (pred n) in loop 4 end
generate a command-line interface specialised to it. Functoria: keys The keys can be supplied at build time, or dynamically at execution time. The more that is supplied at build time, the better the application gets!
(kv_ro @-> job) let () = register "kv_ro" [main $ disk] Ways to build to implement a key/value store: • Crunch the files directly into the binary. No external devices needed, but all the files need to be in memory. • Passthrough to an underlying Unix filesystem. • Construct a key/value device from an arbitrary filesystem implementation. • Tar an archive from an underlying block device.
(kv_ro @-> job) let () = register "kv_ro" [main $ disk] $ mirage describe -t xen —dot Crunch Tar Mmap No option for a “passthrough” filesystem in Xen mode as there is no Unix operating system available!
(kv_ro @-> job) let () = register "kv_ro" [main $ disk] $ mirage describe —dot Some options must be specified at build time (e.g. the hardware target) Others can remain dynamic if desired, at the expense of less specialisation
= register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] Let’s consider an example with networking. This can be even more complicated, with many ways to configure all the various pieces of the stack. Functoria absolutely shines here! Let’s look at a static web server…
= register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] $ mirage describe -t unix --net=socket --dhcp=false --dot This binary hooks into the Unix socket stack and is just like a normal application
= register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] $ mirage describe -t xen --dot In Xen, we need to hook up an entire networking stack from scratch, since there is no OS support
= register "conduit_server" [ handler $ conduit_direct (generic_stackv4 default_network) ] $ mirage describe -t xen --dot In Xen, we need to hook up an entire networking stack from scratch, since there is no OS support No worries! We have written an entire TCP/IP stack in this functor style
the “shapes” of the nodes using types, so that a network stack cannot accidentally become a storage device. • Incrementally specialises the graph as more information becomes available. • Generates code to turn a configuration set into a full unikernel. Functoria: recap
The signatures are easily extensible as you just create a new one for a specific domain. • This is how we break the operating cycle of bloat. • Design modular interfaces that are fit for a purpose and no more. Functoria
Effects for sequential but concurrent code - no more sort-of-monads • Staging to eliminate configurations and do build time specialisation. Key Things We Need to Replace All the Bad OS Things ✓ ✓ ?
avsm
iselegant
kimujun
smt7174
oracle4engineer
non97
taddy_919
gimupop
cyberagentdevelopers
calm1205
bengo4com
shotashiratori
huffon
mongodb
aki_iinuma
jakevdp
reverentgeek
kevinliebholz
speakerdeck
lara
keavy
jponch
bkeepers
sferik
jmmastey
