Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Leveraging Rust with mruby: Loving our fellow W...

hone
February 04, 2017

Leveraging Rust with mruby: Loving our fellow Windows® users - FOSDEM 2017

Even if mruby core works on Windows, any mruby app is may not work if the libraries it depends on don’t compile and run on Windows. Using Rust, we can work towards building an ecosystem that is Windows compatible. This talk walks through building mruby libraries that do that.

Unlike some other languages like JavaScript, Java, or Python, Ruby is not known for it’s ability to run well on Windows. The Windows environment is often treated as a second or third class citizen even though there are many potential users. Part of this is due to Ruby’s strong POSIX oriented standard library. This has mostly been ok since most Ruby applications are hosted and run on UNIX servers.

mruby is a lightweight Ruby that can be linked and embedded inside other applications. Being embeddable means that the runtime host could be Windows. mruby also has no Operating System specific standard library. Since the ecosystem is young, there’s potential for an ecosystem that is Windows friendly. But writing cross platform native extensions in C for mruby is still difficult.

Rust is a modern systems programming language that is fast and built to prevent segfaults and guarantee thread safety. It’s more than just a compiler, it’s a platform that has a great dependency manager and cares about cross compilation out of the box. This makes it a great candidate over C to write native extensions.

In this talk, we’ll go over some basic Rust and how we can integrate this to write mruby libraries that compile on Linux, OS X, and Windows. It’ll dig into not only writing the library code, but also the mruby C API as well as the mruby/Rust binding layer that makes this all possible.

Video: https://fosdem.org/2017/schedule/event/ruby_leverging_rust_to_build_a_windows_friendly_ecosystem_for_mruby/

hone

February 04, 2017
Tweet

More Decks by hone

Other Decks in Programming

Transcript

  1. Agenda • Ruby on Windows • Intro to mruby &

    rust • mruby + rust • Future?
  2. Installing Ruby on Windows • chruby ◦ Requirements: bash >=

    3 or zsh • RVM ◦ https://rvm.io/support/faq#does-rvm-work-on-windows-will-it-in-the-future ◦ Does RVM work on windows? Will it in the future? ▪ NO. If you would like to manage multiple versions of ruby on windows please use pik which is an excellent tool by Gordon Thiesfeld. You can find it on GitHub. There are plans to include windows support in RVM 2.0 => https://www.bountysource.com/fundraisers/489-rvm-2-0. • ruby-build ◦ bash script
  3. mruby is the lightweight implementation of the Ruby language complying

    with part of the ISO standard. mruby can be linked and embedded within your application.
  4. mrbgem.rake MRuby::Gem::Specification.new('hello') do |spec| spec.license = 'MIT' spec.authors = ['Terence

    Lee'] spec.summary = 'hello world' spec.bins = ['hello'] spec.add_dependency 'mruby-print', :core => 'mruby-print' spec.add_dependency 'mruby-mtest', :mgem => 'mruby-mtest' spec.add_dependency 'mruby-yaml', :github => 'hone/mruby-yaml' end
  5. core mrbgems ├── [4.0K] mruby-array-ext ├── [4.0K] mruby-bin-debugger ├── [4.0K]

    mruby-bin-mirb ├── [4.0K] mruby-bin-mrbc ├── [4.0K] mruby-bin-mruby ├── [4.0K] mruby-bin-mruby-config ├── [4.0K] mruby-bin-strip ├── [4.0K] mruby-compiler ├── [4.0K] mruby-enumerator
  6. Rust is a systems programming language that runs blazingly fast,

    prevents segfaults, and guarantees thread safety.
  7. Zero Cost Abstractions pub fn main() { let array =

    [1,2,3]; let mut sum = 0; for i in 0..3 { sum += array[i] } println!("{:?}", sum); }
  8. Zero Cost Abstractions pub fn main() { let array =

    [1,2,3]; let sum = array.iter().fold(0, |sum, &val| sum + val); println!("{:?}", sum); }
  9. Rust 1.0: Stability as a Deliverable (2014) https://blog.rust-lang.org/2014/10/30/Stability.html 6 week

    release cycle Code compiles on Rust stable 1.0 should compile with Rust stable 1.x
  10. Rust Cross-compilation https://blog.rust-lang.org/2016/05/13/rustup.html rustup - a toolchain manager for Rust.

    "rustup works the same on Windows as it does on Unix" - rustup README Targets: $ rustup target list | wc -l 36 Including: • x86_64-apple-darwin (installed) • x86_64-pc-windows-gnu (installed) • x86_64-unknown-linux-gnu (default) • x86_64-unknown-linux-musl (installed)
  11. Shipping Rust in Firefox https://hacks.mozilla.org/2016/07/shipping-rust-in-firefox/ "Starting with Firefox 48, Mozilla

    is shipping its first production Rust code, with more to come!" -Dave Herman, Director of Strategy at Mozilla Research
  12. Cargo.toml [package] name = "hello_world" version = "0.1.0" authors =

    ["Your Name <[email protected]>"] [dependencies] time = "0.1.12" regex = "0.1.41"
  13. cargo build $ cargo build Updating registry `https://github.com/rust-lang/crates.io-index` Downloading memchr

    v0.1.5 Downloading libc v0.1.10 Downloading regex-syntax v0.2.1 Downloading memchr v0.1.5 Downloading aho-corasick v0.3.0 Downloading regex v0.1.41 Compiling memchr v0.1.5 Compiling libc v0.1.10 Compiling regex-syntax v0.2.1 Compiling memchr v0.1.5 Compiling aho-corasick v0.3.0 Compiling regex v0.1.41 Compiling hello_world v0.1.0 (file:///path/to/project/hello_world)
  14. Cargo.lock [root] name = "hello_world" version = "0.1.0" dependencies =

    [ "regex 0.1.80 (registry+https://github.com/rust-lang/crates.io-index)", "time 0.1.36 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "aho-corasick" version = "0.5.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "memchr 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)", ]
  15. ├── [4.0K] bintest │ └── [ 437] mruby-hello-world.rb ├── [2.3K]

    build_config.rb ├── [ 298] mrbgem.rake ├── [4.0K] mrblib │ └── [ 42] mruby-hello-world.rb ├── [1.9K] Rakefile ├── [4.0K] src │ └── [ 363] hello_world_gem.c ├── [4.0K] test │ └── [ 119] test_mruby-hello-world.rb └── [4.0K] tools └── [4.0K] mruby-hello-world └── [ 755] mruby-hello-world.c
  16. ├── [4.0K] bintest │ └── [ 437] mruby-hello-world.rb ├── [2.3K]

    build_config.rb ├── [ 298] mrbgem.rake ├── [4.0K] mrblib │ └── [ 42] mruby-hello-world.rb ├── [1.9K] Rakefile ├── [4.0K] src │ └── [ 363] hello_world_gem.c ├── [4.0K] test │ └── [ 119] test_mruby-hello-world.rb └── [4.0K] tools └── [4.0K] mruby-hello-world └── [ 755] mruby-hello-world.c
  17. hello_world_gem.c #include "mruby.h" mrb_value c_hello(mrb_state * mrb, mrb_value self) {

    return mrb_str_new_cstr(mrb, "Hello"); } void mrb_mruby_hello_world_gem_init(mrb_state * mrb) { struct RClass *klass = mrb_define_module(mrb, "Hello"); mrb_define_class_method(mrb, klass, "hello", c_hello, MRB_ARGS_NONE()); } void mrb_mruby_hello_world_gem_final(mrb_state * mrb) { }
  18. hello_world_gem.c #include "mruby.h" mrb_value c_hello(mrb_state * mrb, mrb_value self) {

    return mrb_str_new_cstr(mrb, "Hello"); } void mrb_mruby_hello_world_gem_init(mrb_state * mrb) { struct RClass *klass = mrb_define_module(mrb, "Hello"); mrb_define_class_method(mrb, klass, "hello", c_hello, MRB_ARGS_NONE()); } void mrb_mruby_hello_world_gem_final(mrb_state * mrb) { }
  19. hello_world_gem.c #include "mruby.h" mrb_value c_hello(mrb_state * mrb, mrb_value self) {

    return mrb_str_new_cstr(mrb, "Hello"); } void mrb_mruby_hello_world_gem_init(mrb_state * mrb) { struct RClass *klass = mrb_define_module(mrb, "Hello"); mrb_define_class_method(mrb, klass, "hello", c_hello, MRB_ARGS_NONE()); } void mrb_mruby_hello_world_gem_final(mrb_state * mrb) { }
  20. hello_world_gem.c #include "mruby.h" mrb_value c_hello(mrb_state * mrb, mrb_value self) {

    return mrb_str_new_cstr(mrb, "Hello"); } void mrb_mruby_hello_world_gem_init(mrb_state * mrb) { struct RClass *klass = mrb_define_module(mrb, "Hello"); mrb_define_class_method(mrb, klass, "hello", c_hello, MRB_ARGS_NONE()); } void mrb_mruby_hello_world_gem_final(mrb_state * mrb) { }
  21. ├── [4.0K] bintest │ └── [ 437] mruby-hello-world.rb ├── [2.3K]

    build_config.rb ├── [ 298] mrbgem.rake ├── [4.0K] mrblib │ └── [ 42] mruby-hello-world.rb ├── [1.9K] Rakefile ├── [4.0K] src │ └── [ 363] hello_world_gem.c ├── [4.0K] test │ └── [ 119] test_mruby-hello-world.rb └── [4.0K] tools └── [4.0K] mruby-hello-world └── [ 755] mruby-hello-world.c
  22. Rust FFI https://blog.rust-lang.org/2015/04/24/Rust-Once-Run-Everywhere.html > Rust makes it easy to communicate

    with C APIs without overhead, and to leverage its ownership system to provide much stronger safety guarantees for those APIs at the same time. #[no_mangle] pub extern "C" fn double_input(input: i32) -> i32 { input * 2 }
  23. ├── [4.0K] bintest │ └── [ 437] mruby-rust-hello-world.rb ├── [2.3K]

    build_config.rb ├── [2.4K] mrbgem.rake ├── [4.0K] mrblib │ ├── [4.0K] mruby-rust-hello-world │ └── [ 41] mruby-rust-hello-world.rb ├── [1.9K] Rakefile ├── [4.0K] rust │ ├── [ 460] Cargo.lock │ ├── [ 196] Cargo.toml │ └── [4.0K] src │ └── [ 710] lib.rs ├── [4.0K] test │ └── [ 119] test_mruby-rust.rb └── [4.0K] tools └── [4.0K] mruby-rust-hello-world └── [ 759] mruby-rust-hello-world.c
  24. lib.rs #[no_mangle] pub extern "C" fn mrb_rust_hello(mrb: *mut mrb_state, selfie:

    mrb_value) -> mrb_value { unsafe { mrb_str_new_cstr(mrb, cstr!("Hello")) } } #[no_mangle] pub extern "C" fn mrb_mruby_rust_hello_world_gem_init(mrb: *mut mrb_state) { unsafe { let klass = mrb_define_module(mrb, cstr!("Rust")); mrb_define_class_method(mrb, klass, cstr!("hello"), mrb_rust_hello as mrb_func_t, MRB_ARGS_NONE()); } } #[no_mangle] pub extern "C" fn mrb_mruby_rust_hello_world_gem_final(mrb: *mut mrb_state) { }
  25. Cargo.toml [package] name = "hello" version = "0.1.0" authors =

    ["Terence Lee <[email protected]>"] [dependencies] mferuby = { git = "https://github.com/hone/mferuby.git" } [lib] crate-type = ["staticlib"]
  26. libmruby-sys crate ├── [4.0K] crates │ ├── [4.0K] libmruby-sys │

    │ ├── [ 150] Cargo.toml │ │ └── [4.0K] src │ │ └── [5.1K] lib.rs │ └── [4.0K] mferuby │ ├── [ 203] Cargo.toml │ └── [4.0K] src │ ├── [ 588] lib.rs │ └── [ 815] mrb.rs low level rust binding to mruby C API
  27. crates/libmruby-sys/src/lib.rs extern { #[link_name = "tmrb_nil_value"] pub fn nil() ->

    mrb_value; #[link_name = "tmrb_true_value"] pub fn mrb_true() -> mrb_value; #[link_name = "tmrb_false_value"] pub fn mrb_false() -> mrb_value; #[link_name = "tmrb_fixnum_value"] pub fn fixnum(i: c_int) -> mrb_value; #[link_name = "tmrb_float_value"] pub fn float(mrb: *mut mrb_state, f: c_double) -> mrb_value; pub fn mrb_open() -> *mut mrb_state; pub fn mrb_define_module(mrb: *mut mrb_state, name: *const c_char) -> *mut RClass; pub fn mrb_define_class(mrb: *mut mrb_state, name: *const c_char, super_class: *mut RClass) -> *mut RClass; pub fn mrb_define_class_under(mrb: *mut mrb_state, outer: *mut RClass, name: *const c_char, super_class: *mut RClass) -> *mut
  28. mferuby crate ├── [4.0K] crates │ ├── [4.0K] libmruby-sys │

    │ ├── [ 150] Cargo.toml │ │ └── [4.0K] src │ │ └── [5.1K] lib.rs │ └── [4.0K] mferuby │ ├── [ 203] Cargo.toml │ └── [4.0K] src │ ├── [ 588] lib.rs │ └── [ 815] mrb.rs high level rust API on top of libmruby-sys
  29. crates/mferuby/src/lib.rs pub fn mruby_str_to_rust_string(mruby_string: sys::mrb_value) -> Result<String, std::str::Utf8Error> { let

    size = unsafe { sys::RSTRING_LEN(mruby_string) }; let ptr = unsafe { sys::RSTRING_PTR(mruby_string) }; let slice = unsafe { std::slice::from_raw_parts(ptr as *const u8, size as usize) }; let s = try!(std::str::from_utf8(slice)); Ok(s.to_string()) }
  30. mferuby-runtime mrbgem ├── [2.3K] build_config.rb ├── [ 412] mrbgem.rake ├──

    [1.9K] Rakefile ├── [4.0K] src │ ├── [1.4K] mferuby_runtime.c │ ├── [ 138] mferuby_runtime_gem.c │ └── [ 37] mruby.rs └── [4.0K] test └── [ 131] test_mferuby-runtime.rb compiles against mruby and exposes non-public C API to rust
  31. src/mferuby_runtime.c #include <mruby/compile.h> #include <mruby/value.h> mrb_value tmrb_float_value(struct mrb_state *mrb, mrb_float

    f) { return mrb_float_value(mrb, f); } mrb_value tmrb_fixnum_value(mrb_int i) { return mrb_fixnum_value(i); } mrb_value tmrb_nil_value() { return mrb_nil_value(); } mrb_value tmrb_true_value() { return mrb_true_value(); }
  32. class CommandLine USAGE = <<USAGE Heroku Docker Ruby Util. Usage:

    ruby-util detect-ruby <gemfile-path> ruby-util detect-gem <profiled-path> <gem> ruby-util install-ruby <gemfile-path> <install-path> <profiled-path> Options: -h --help Show this message USAGE def run(argv) options = Docopt.parse(USAGE, argv) if options["detect-ruby"] puts CLI::DetectRuby.new(File.read(options["<gemfile-path>"])).ruby_ver sion
  33. mruby-docopt pub extern "C" fn parse(mrb: *mut sys::mrb_state, this: sys::mrb_value)

    -> sys::mrb_value { let mrb_obj = mferuby::Mrb::new(mrb); let mut usage: sys::mrb_value = unsafe { mem::uninitialized() }; let mut argv: sys::mrb_value = unsafe { mem::uninitialized() }; unsafe { sys::mrb_get_args(mrb, cstr!("SA"), &mut usage, &mut argv); } let rust_usage = mferuby::mruby_str_to_rust_string(usage).unwrap(); let argc = unsafe { sys::RARRAY_LEN(argv) }; let mut vec_args: Vec<String> = vec![]; for i in 0..argc { let element = unsafe { sys::mrb_ary_ref(mrb, argv, i) }; vec_args.push(mferuby::mruby_str_to_rust_string(element).unwrap()) ;
  34. mruby-docopt pub extern "C" fn parse(mrb: *mut sys::mrb_state, this: sys::mrb_value)

    -> sys::mrb_value { let mrb_obj = mferuby::Mrb::new(mrb); let mut usage: sys::mrb_value = unsafe { mem::uninitialized() }; let mut argv: sys::mrb_value = unsafe { mem::uninitialized() }; unsafe { sys::mrb_get_args(mrb, cstr!("SA"), &mut usage, &mut argv); } let rust_usage = mferuby::mruby_str_to_rust_string(usage).unwrap(); let argc = unsafe { sys::RARRAY_LEN(argv) }; let mut vec_args: Vec<String> = vec![]; for i in 0..argc { let element = unsafe { sys::mrb_ary_ref(mrb, argv, i) }; vec_args.push(mferuby::mruby_str_to_rust_string(element).unwrap()) ;
  35. mruby-docopt pub extern "C" fn parse(mrb: *mut sys::mrb_state, this: sys::mrb_value)

    -> sys::mrb_value { let mrb_obj = mferuby::Mrb::new(mrb); let mut usage: sys::mrb_value = unsafe { mem::uninitialized() }; let mut argv: sys::mrb_value = unsafe { mem::uninitialized() }; unsafe { sys::mrb_get_args(mrb, cstr!("SA"), &mut usage, &mut argv); } let rust_usage = mferuby::mruby_str_to_rust_string(usage).unwrap(); let argc = unsafe { sys::RARRAY_LEN(argv) }; let mut vec_args: Vec<String> = vec![]; for i in 0..argc { let element = unsafe { sys::mrb_ary_ref(mrb, argv, i) }; vec_args.push(mferuby::mruby_str_to_rust_string(element).unwrap()) ;
  36. let result = docopt::Docopt::new(rust_usage) .and_then(|d| d.help(false).argv(vec_args.into_iter()).parse()); match result { Ok(args)

    => { let args = Box::new(args); let klass = unsafe { sys::mrb_class_get_under(mrb, sys::mrb_module_get(mrb, cstr!("Docopt")), cstr!("Options")) }; unsafe { sys::mrb_obj_value(mrb_obj.data_object_alloc::<docopt::ArgvMap>(kl ass, args, &docopt_option_type)) } }, Err(e) => unsafe { println!("ERROR: {:?}", e); sys::nil() }, } }
  37. let result = docopt::Docopt::new(rust_usage) .and_then(|d| d.help(false).argv(vec_args.into_iter()).parse()); match result { Ok(args)

    => { let args = Box::new(args); let klass = unsafe { sys::mrb_class_get_under(mrb, sys::mrb_module_get(mrb, cstr!("Docopt")), cstr!("Options")) }; unsafe { sys::mrb_obj_value(mrb_obj.data_object_alloc::<docopt::ArgvMap>(kl ass, args, &docopt_option_type)) } }, Err(e) => unsafe { println!("ERROR: {:?}", e); sys::nil() }, } }
  38. let result = docopt::Docopt::new(rust_usage) .and_then(|d| d.help(false).argv(vec_args.into_iter()).parse()); match result { Ok(args)

    => { let args = Box::new(args); let klass = unsafe { sys::mrb_class_get_under(mrb, sys::mrb_module_get(mrb, cstr!("Docopt")), cstr!("Options")) }; unsafe { sys::mrb_obj_value(mrb_obj.data_object_alloc::<docopt::ArgvMap>(kl ass, args, &docopt_option_type)) } }, Err(e) => unsafe { println!("ERROR: {:?}", e); sys::nil() }, } }
  39. class CommandLine USAGE = <<USAGE Heroku Docker Ruby Util. Usage:

    ruby-util detect-ruby <gemfile-path> ruby-util detect-gem <profiled-path> <gem> ruby-util install-ruby <gemfile-path> <install-path> <profiled-path> Options: -h --help Show this message USAGE def run(argv) options = Docopt.parse(USAGE, argv) if options["detect-ruby"] puts CLI::DetectRuby.new(File.read(options["<gemfile-path>"])).ruby_ver sion
  40. mruby-docopt in the wild --- a/mrbgem.rake +++ b/mrbgem.rake @@ -12,6

    +12,7 @@ MRuby::Gem::Specification.new('heroku-docker-ruby-util') do |spec| spec.add_dependency 'mruby-stringio', github: 'ksss/mruby-stringio' spec.add_dependency 'mruby-process', github: 'hone/mruby-process', branch: 'header' spec.add_dependency 'mruby-io', github: 'hone/mruby-io', branch: 'popen_status' + spec.add_dependency 'mruby-docopt', github: 'hone/mruby-docopt', branch: 'rust' # test deps spec.add_dependency 'mruby-time', core: 'mruby-time'
  41. mferuby • Cover more mruby C API with Rust binding

    • Create higher level abstractions • Using mferuby shouldn't require "unsafe"