Share
Embed
Evaluate Ruby Without Ruby
RubyConf 2016 at Cincinnati
http://rubyconf.org
Takashi Kokubun
November 10, 2016
Transcript
57 slides
-
Evaluate Ruby Without Ruby Takashi Kokubun / Cookpad Inc. November
10th, 2016 RubyConf 2016 -
Agenda • What’s Ruby DSL? • Why did we want
to evaluate Ruby DSL without MRI? • Introduction to mruby • 2 ways to build portable CLI apps configured by Ruby DSL -
What’s Ruby DSL? What’s Ruby DSL? Why Ruby DSL without
MRI? Introduction to mruby Building portable CLI apps -
Ruby DSL • Domain Specific Language is a language specialized
to a particular application domain • “Ruby DSL” is a DSL defined in terms of Ruby language -
Example Ruby DSL for server provisioning package 'nginx' do action
:install end service 'nginx' do action [:enable, :start] end -
Example Ruby DSL for server provisioning package 'nginx' do action
:install end service 'nginx' do action [:enable, :start] end CentOS yum -y install nginx Ubuntu apt-get -y install nginx -
Example Ruby DSL for server provisioning package 'nginx' do action
:install end service 'nginx' do action [:enable, :start] end CentOS chkconfig nginx on service nginx start Ubuntu systemctl enable nginx systemctl start nginx -
Why Ruby DSL? package 'nginx' do action :install end service
'nginx' do action [:enable, :start] end • Abstraction for a domain- specific purpose • Happy to write Ruby! • It’s boring to write this in JSON -
How is Ruby DSL implemented? Expected behavior script = <<-EOS
package 'nginx' EOS dsl = DSL.new dsl.instance_eval(script) dsl.result #=> [["package", "nginx"]] -
How is Ruby DSL implemented? Expected behavior Implementation class DSL
def package(name, &block) result << ['package', name] end def result; @result ||= []; end end script = <<-EOS package 'nginx' EOS dsl = DSL.new dsl.instance_eval(script) dsl.result #=> [["package", "nginx"]] -
How is Ruby DSL implemented? Expected behavior class DSL def
package(name, &block) result << ['package', name] end def result; @result ||= []; end end Implementation script = <<-EOS package 'nginx' do action :install end EOS dsl = DSL.new dsl.instance_eval(script) dsl.result #=> [["package", "nginx", {:action=>[:install]}]] -
How is Ruby DSL implemented? Expected behavior class DSL def
package(name, &block) pkg = Package.new pkg.instance_exec(&block) result << ['package', name, pkg.attrs] end def result; @result ||= []; end end class Package def action(actions) attrs[:action] = Array(actions) end def attrs; @attrs ||= {}; end end Implementation script = <<-EOS package 'nginx' do action :install end EOS dsl = DSL.new dsl.instance_eval(script) dsl.result #=> [["package", "nginx", {:action=>[:install]}]] -
How is Ruby DSL implemented? • Usually Ruby DSL is
implemented with MRI • Using instance_eval, instance_exec, …etc • But we wanted to evaluate Ruby DSL without MRI • Why? Let’s have a look. -
Why Ruby DSL without MRI? What’s Ruby DSL? Why Ruby
DSL without MRI? Introduction to mruby Building portable CLI apps -
Ruby DSL use case at Cookpad • Development environment bootstrap
for macOS and Linux • We wanted to use Ruby DSL to simply describe operations for multiple OS distributions • For maintenance cost, we used the same Ruby DSL as a server provisioning tool we develop and use, which is Itamae • https://github.com/itamae-kitchen/itamae -
Ruby DSL use case at Cookpad package 'memcached' package 'postgresql'
macOS brew install memcached brew install postgresql -
Ruby DSL use case at Cookpad package 'memcached' package 'postgresql'
macOS Ubuntu apt-get install -y memcached apt-get install -y postgresql brew install memcached brew install postgresql -
Problem to bootstrap development environment using MRI • There are
difficulties to use MRI on an initial environment • In macOS, we don’t want to use end-of-life Ruby, which is shipped with latest macOS • In Linux, most Linux distro doesn't ship MRI out of the box • We don’t want to manually install MRI prior to bootstrap -
Problem to bootstrap development environment using MRI • If you
use MRI for user’s environment, you have to care about how a gem is set up everytime you run Ruby DSL • Are rbenv and MRIs already installed? • Is the gem installed to MRI currently selected by rbenv? • Packaging ruby, gem, … to run just one gem is a little overkill -
How can we remove dependency of MRI? • All these
problems are resolved if it doesn’t depend on MRI • Can we embed Ruby VM into our software? • OK, it’s time to use mruby! -
Introduction to mruby What’s Ruby DSL? Why Ruby DSL without
MRI? Introduction to mruby Building portable CLI apps -
mruby • Lightweight implementation of Ruby • Embeddable in: •
Hardware • Software -
Embedding mruby into software • You can evaluate Ruby scripts
from any software by embedding mruby • It allows us to build a software evaluating Ruby DSL and independent to MRI -
How to embed and use mruby 1. git clone https://github.com/mruby/mruby;
cd mruby; make 2. Link mruby/build/host/lib/libmruby.a to your software 3. Call mruby’s C API • Or use its binding for a language you use -
How to embed and use mruby #include <stdio.h> #include <mruby.h>
#include <mruby/compile.h> #include <mruby/string.h> int main(void) { mrb_state *mrb = mrb_open(); mrb_value str = mrb_load_string(mrb, "%w[hello world].join"); // Prints "helloworld\n" printf("%s\n", mrb_string_value_cstr(mrb, &str)); return 0; } -
Now we can embed mruby into our software. Is it
enough? • No, our use case is development environment bootstrap and we should also care about dependencies other than MRI • We want to build it as a “single binary” • There are some good ways to do that -
Building portable CLI apps configured by Ruby DSL What’s Ruby
DSL? Why Ruby DSL without MRI? Introduction to mruby Building portable CLI apps -
2 ways to build portable CLI apps configured by Ruby
DSL • go-mruby • Use mruby only for evaluating Ruby DSL and implement others in Go language • mruby-cli • Implement both Ruby DSL and others using mruby -
2 ways to build portable CLI apps configured by Ruby
DSL • I experimentally re-implemented our configuration management tool “Itamae” in both ways • Let’s compare these methods! -
Example 1: Itamae Go • Itamae Go • Alternative implementation
of “Itamae”, a configuration management tool inspired by Chef • Single binary, built with Go language and go-mruby • https://github.com/k0kubun/itamae-go -
How to run Ruby scripts from Go 1. git clone
https://github.com/mruby/mruby; cd mruby; make 2. git clone https://github.com/mitchellh/go-mruby; cp /path/to/mruby/build/host/libmruby.a ./go-mruby 3. import "github.com/mitchellh/go-mruby" and call mruby APIs -
// Evaluate Ruby from Go (mitchellh/go-mruby) mrb := mruby.NewMrb() //
Call mrb_load_string str := mrb.LoadString("%w[hello world].join") // Prints "helloworld\n" fmt.Println(mrb.StringValue(str)) How to run Ruby scripts from Go -
How to implement Ruby DSL in Go • To implement
IO operations in Go, it is necessary to get data for Go from an evaluation result of Ruby DSL • I implemented method for Ruby DSL in Go language -
How to implement Ruby DSL in Go func Package(mrb *mruby.Mrb,
self *mruby.MrbValue) (mruby.Value, mruby.Value) { return mruby.Nil, nil } // In another place cDSL.DefineMethod( “package", Package, mruby.ArgsReq(1)) -
How to implement Ruby DSL in Go func Package(mrb *mruby.Mrb,
self *mruby.MrbValue) (mruby.Value, mruby.Value) { args := mrb.GetArgs() pkg := PackageResource{Name: args[0].String()} resources = append(resources, pkg) return mruby.Nil, nil } // In another place cDSL.DefineMethod( “package", Package, mruby.ArgsReq(1)) type PackageResource struct { Name string } -
How to implement Ruby DSL in Go func Package(mrb *mruby.Mrb,
self *mruby.MrbValue) (mruby.Value, mruby.Value) { args := mrb.GetArgs() pkg := PackageResource{Name: args[0].String()} if len(args) > 1 { context, _ := mrb.LoadString("Package.new") context.CallBlock("instance_eval", args[1]) attrs, _ := context.Call("attributes") action, _ := mrb.LoadString(":action") value, _ := attrs.Call("[]", action) pkg.Action = value.Array() } resources = append(resources, pkg) return mruby.Nil, nil } // In another place cDSL.DefineMethod( “package", Package, mruby.ArgsReq(1)) type PackageResource struct { Name string Action []string } -
It was a little hard to maintain • Writing bridge
to connect Ruby and Go world is really boring • Checking errors for all Ruby method calls • Required to use 3 languages: Ruby, Go, C (for mrbgem) • You need to prepare environment for cross compiling by yourself -
Why did we need Go language? • Purposes • To
make it as a single binary • To implement IO operations for development environment bootstrap • Can't we achieve them without Go language? • Try mruby-cli! -
Example 2: MItamae • MItamae • Alternative implementation of Itamae
and Itamae Go, which is actively maintained • Single binary, built with mruby-cli and written in pure Ruby • https://github.com/k0kubun/mitamae -
What is mruby-cli? • A cross-compile platform to build CLI
apps using mruby • https://github.com/hone/mruby-cli • Eric gave a talk "Building maintainable command-line tools with MRuby" this morning • If you've missed it, watch the video later -
How to use mruby-cli 1. Download mruby-cli command from https://github.com/hone/mruby-cli/releases
2. Generate boilerplate by mruby-cli -s app_name 3. cd app_name; docker-compose run compile • You can get binaries in ./mruby/build/*/bin/app_name -
How to implement Ruby DSL with mruby-cli • Almost the
same way as MRI • Required to add some “mrbgems” to call methods which is not provided by default • mruby-eval for instance_eval • mruby-object-ext for instance_exec -
What is mrbgem? • It’s like rubygem for MRI •
Unlike rubygem, it’s compiled together when building mruby • So we can use it without adding dependency to a binary • Some of built-in features in MRI are separated to mrbgem in mruby -
Porting existing stdlibs and rubygems to mrbgems • It’s easy
when they are written in Ruby • Most features required for Ruby programming except keyword arguments are available at existing mrbgems • mrbgems I created to implement MItamae: • mruby-shellwords • mruby-hashie -
Implementing IO operations with mruby • There are already IO.popen
and Kernel.` (`command`) • But I wanted to capture stdout, stderr and status separately • There were no Open3 and Kernel.#spawn • It was necessary to write C to implement Kernel.#spawn -
Incomplete implementation of spawn to implement Open3.capture3 static mrb_value mrb_open3_spawn(mrb_state
*mrb, mrb_value self) { const char **cmd; pid_t pid; mrb_value *argv; mrb_int argc, out_dst, err_dst; mrb_get_args(mrb, "*", &argv, &argc); cmd = mrb_str_buf_to_cstr_buf(mrb, argv, argc-1); open3_spawn_process_options(mrb, argv[argc-1], &out_dst, &err_dst); pid = fork(); if (pid == 0) { dup2(out_dst, STDOUT_FILENO); dup2(err_dst, STDERR_FILENO); execvp(cmd[0], cmd); } return mrb_fixnum_value(pid); } -
Finally I could built mruby-open3 that has only Open3.capture3 module
Open3 # @param [Array<String>] - command to execute # @return [String, String, Process::Status] - stdout, stderr, status def capture3(*cmd) out_r, out_w = IO.pipe err_r, err_w = IO.pipe pid = spawn(*cmd, { 'out' => out_w.to_i, 'err' => err_w.to_i }) # … snip … _, status = Process.waitpid2(pid) [stdout, stderr, status] end module_function :capture3 end -
Isn’t it hard to implement apps with mruby-cli? • Yes,
it’s hard a little when necessary mrbgems are absent • But there are great merits to build apps using mruby-cli -
Merits to build CLI apps using mruby-cli • You can
distribute apps easily • Not depending on users' environment • Cross-compilable -
Merits to build CLI apps using mruby-cli • Fast startup
on command execution $ time itamae version Itamae v1.9.9 itamae version 0.54s user 0.22s system 98% cpu 0.776 total $ time mitamae version MItamae v0.6.0 mitamae version 0.02s user 0.00s system 85% cpu 0.023 total on 1.3 GHz Intel Core m7 -
Merits to build CLI apps using mruby-cli • Maintainable •
No bridging code • Happy to write all implementation in Ruby! -
Conclusion • To simplify development environment bootstrap, I made a
configuration management tool configured by Ruby DSL as a single binary using mruby • https://github.com/k0kubun/mitamae • There are 2 ways to build a tool configured by Ruby DSL as a single binary • If there’s a reason to use Go language, use go-mruby • Otherwise mruby-cli is recommended