Helix: Native Extensions for Everyone - RubyConf Colombia 2017

You can write a native extension for Ruby using Rust through Helix. With Helix your code will be screaming fast, while ensuring thread and memory safety and preventing segfaults. Fast, reliable, and productive. Come learn how we can have our cake and eat it too!

Writing Faster Ruby: https://speakerdeck.com/sferik/writing-fast-ruby

1. Native Extensions for Everyone
2. Terence Lee @hone02
5. Austin, TX
6. Austin, TX
7. Native Extensions for Everyone
8. Ruby is Awesome • Beautiful syntax • Optimize for happiness • Community • Mature Ecosystem
9. Ruby is Slow • Usually it doesn't matter • Most workloads are I/O bound • But occasionally it does...
10. "Best of both worlds" • Native extensions • JSON gem, nokogiri gem, etc…. • stdlib too: Date, Pathname, etc... • Very fast • Transparent to the user
11. Case Study: Discourse • String#blank? Is a hotspot for Discourse • Sam Saffron's fast_blank • 50 LoC in C • 20x speedup of String#blank?
12. Why don't we do it more?
13. But C... • Unsafe • Risky - segfaults! • Maintenance burden • Contribution barrier • Cross Platform Support
14. Case Study #2: New Relic RPM • Monitoring agent built in Ruby • Hard to minimize overhead blog.scoutapp.com/articles/2016/02/07/overhead-benchmarks-new-relic-vs-scout
15. Case Study #3: Skylight Agent • Started with Ruby • Too much overhead • Native extension! • But C… • AppSignal has written parts of their agent in Rust too
16. Meet Rust
17. Meet Rust • Like C: compiled, statically typed, very fast • Unlike C: enjoyable to use, guarantees safety • "If it compiles, it doesn't crash" • Same guarantee as Ruby with memory safety • Multiplatform Support
18. Cargo - Dependency Manager
19. Cargo.toml [package] name = "hello_world" version = "0.1.0" authors = ["Your Name <you@example.com>"] [dependencies] time = "0.1.12" regex = "0.1.41"
20. cargo build $ cargo build Updating registry `https://github.com/rust-lang/crates.io-index` Downloading memchr v0.1.5 Downloading libc v0.1.10 Downloading regex-syntax v0.2.1 Downloading memchr v0.1.5 Downloading aho-corasick v0.3.0 Downloading regex v0.1.41 Compiling memchr v0.1.5 Compiling libc v0.1.10 Compiling regex-syntax v0.2.1 Compiling memchr v0.1.5 Compiling aho-corasick v0.3.0 Compiling regex v0.1.41 Compiling hello_world v0.1.0 (file:///path/to/project/hello_world)
21. Cargo.lock [root] name = "hello_world" version = "0.1.0" dependencies = [ "regex 0.1.80 (registry+https://github.com/rust-lang/crates.io-index)", "time 0.1.36 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "aho-corasick" version = "0.5.3" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "memchr 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)", ]
22. Rust Multiplatform Support https://blog.rust-lang.org/2016/05/13/rustup.html rustup - a toolchain manager for Rust. "rustup works the same on Windows as it does on Unix" - rustup README Targets: $ rustup target list | wc -l 36 Including: • x86_64-apple-darwin (installed) • x86_64-pc-windows-gnu (installed) • x86_64-unknown-linux-gnu (default) • x86_64-unknown-linux-musl (installed)
23. Shipping Rust in Firefox https://hacks.mozilla.org/2016/07/shipping-rust-in-firefox/ "Starting with Firefox 48, Mozilla is shipping its first production Rust code, with more to come!" -Dave Herman, Director of Strategy at Mozilla Research
24. Zero-cost abstractions™ • In Ruby: tension between abstractions and performance • hash.keys.each vs hash.each_key • array.shuffle.first vs array.sample • In Rust: no such tradeoff • Compiler is magic
25. Zero-cost abstractions™ pub fn main() { let array = [1,2,3]; let mut sum = 0; for i in 0..3 { sum += array[i] } println!("{:?}", sum); }
26. Zero-cost abstractions™ pub fn main() { let array = [1,2,3]; let sum = array.iter().fold(0, |sum, &val| sum + val); println!("{:?}", sum); }
27. fast_blank in Rust* pub extern "C" in fast_blank(buf: Buf) -> bool { buf.as_slice().chars().all(|c| c.is_whitespace()) } * without boilerplate code
29. fast_blank in Helix #[macro_use] extern crate helix; declare_types! { reopen class RubyString { def is_blank(self) -> bool { self.chars().all(|c| c.is_whitespace()) } } }
30. The Vision • Keep writing the Ruby you love… • ...without the fear of eventually hitting a wall • Start with Ruby • Move to Helix when appropriate
31. Build a Helix Rails App
32. 1. Generate Rails App $ rails new helix-flipper --skip-active-record
33. 2. Add helix-rails dependency Gemfile: gem 'helix-rails', '~> 0.5.0' $ bundle install
34. 3. Rails Generator $ bin/rails generate helix:crate text_transform Running via Spring preloader in process 24825 create crates/text_transform/Cargo.toml create crates/text_transform/src/lib.rs create crates/text_transform/text_transform.gemspec create crates/text_transform/Gemfile create crates/text_transform/lib/tasks/helix_runtime.rake create crates/text_transform/lib/text_transform.rb create crates/text_transform/.gitignore create crates/text_transform/Rakefile append crates/text_transform/Rakefile run bundle from "./crates/text_transform" append Gemfile
35. 3. Rails Generator $ bin/rails generate helix:crate text_transform Running via Spring preloader in process 24825 create crates/text_transform/Cargo.toml create crates/text_transform/src/lib.rs create crates/text_transform/text_transform.gemspec create crates/text_transform/Gemfile create crates/text_transform/lib/tasks/helix_runtime.rake create crates/text_transform/lib/text_transform.rb create crates/text_transform/.gitignore create crates/text_transform/Rakefile append crates/text_transform/Rakefile run bundle from "./crates/text_transform" append Gemfile
36. crates/text_transform/src/lib.rs #[macro_use] extern crate helix; ruby! { class TextTransform { def hello() { println!("Hello from text_transform!"); } } }
37. $ bundle exec rake build $ bundle exec irb > require 'text_transform' => true > TextTransform.hello Hello from text_transform! => nil
38. 4. Implement flip method def flip(text: String) -> String { text.chars().rev().map(|char| { match char { '!' => '¡', '"' => '„', '&' => '⅋', '\'' => '‚', '(' => ')', ')' => '(', ',' => '‘', '.' => '˙', ... // Flip back ... _ => char, } }).collect() end
39. 4. Implement flip method def flip(text: String) -> String { text.chars().rev().map(|char| { match char { '!' => '¡', '"' => '„', '&' => '⅋', '\'' => '‚', '(' => ')', ')' => '(', ',' => '‘', '.' => '˙', ... // Flip back ... _ => char, } }).collect() end
40. 4. Implement flip method def flip(text: String) -> String { text.chars().rev().map(|char| { match char { '!' => '¡', '"' => '„', '&' => '⅋', '\'' => '‚', '(' => ')', ')' => '(', ',' => '‘', '.' => '˙', ... // Flip back ... _ => char, } }).collect() end
41. crates/text_transform/spec/text_transform.rb require "text_transform" describe "TextTransform" do it "can flip text" do expect(TextTransform.flip("Hello Aaron (@tenderlove)!")).to eq("¡(ǝʌolɹǝpuǝʇ@) uoɹɐ∀ ollǝH") end it "can flip the text back" do expect(TextTransform.flip("¡(ǝʌolɹǝpuǝʇ@) uoɹɐ∀ ollǝH")).to eq("Hello Aaron (@tenderlove)!") end end
42. 5. Flipping Tables! def flip(text: String) -> String { if text == "┬──┬ ノ( ゜-゜ノ)" { return "(╯°□°）╯︵ ┻━┻".to_string(); } else if text == "(╯°□°）╯︵ ┻━┻" { return "┬──┬ ノ( ゜-゜ノ)".to_string(); } // rest of method … end
43. 6. Using in Rails $ bin/rails generate helix:crate text_transform Running via Spring preloader in process 24825 create crates/text_transform/Cargo.toml create crates/text_transform/src/lib.rs create crates/text_transform/text_transform.gemspec create crates/text_transform/Gemfile create crates/text_transform/lib/tasks/helix_runtime.rake create crates/text_transform/lib/text_transform.rb create crates/text_transform/.gitignore create crates/text_transform/Rakefile append crates/text_transform/Rakefile run bundle from "./crates/text_transform" append Gemfile
44. 6. Using in Rails Gemfile: gem 'text_transform', path: 'crates/text_transform'
45. 6. Using in Rails (Routes) config/routes.rb: resources :flips, path: "/", only: [:index, :create]
46. 6. Using in Rails (Controller) app/controllers/flips_controller.rb: class FlipsController < ApplicationController def index @text = params[:text] || "Hello world!" end def create @text = TextTransform.flip(params[:text]) render :index end end
47. 6. Using in Rails (View) app/views/flips/index.html.erb: <h1>Flipper</h1> <%= form_tag do %> <%= text_field_tag :text, @text %> <%= submit_tag "Flip!" %> <% end %>
48. 7. Deploy to Heroku Setup rustc on production machine $ heroku create $ heroku buildpacks:add https://github.com/emk/heroku-buildpack-rust $ heroku buildpacks:add heroku/ruby $ git push heroku master
49. 7. Deploy to Heroku Setup rustc on production machine $ heroku create $ heroku buildpacks:add https://github.com/emk/heroku-buildpack-rust $ heroku buildpacks:add heroku/ruby $ git push heroku master
50. Distribution (Precompile Binaries) • Use CI to build binaries and publish to rubygems ◦ Travis/CircleCI: Linux + OS X ◦ Appveyor: Windows • Don't need rust compiler on host machine • Next version of helix CLI will generate CI config ◦ helix bootstrap hello_helix ◦ Add env var RUBYGEMS_AUTH_TOKEN to CI services to publish
51. 7. Extract text_transform gem
52. 7. Extract text_transform gem Gemfile: gem 'text_transform' $ bundle install
53. 8. Deploy to Heroku $ heroku create $ git push heroku master -----> Ruby app detected -----> Compiling Ruby/Rails -----> Using Ruby version: ruby-2.3.4 -----> Installing dependencies using bundler 1.15.2 … Installing text_transform 0.1.0 (x86_64-linux)
54. Table Flip!
55. Table Flip!
56. Whoops… It worked!™
57. Good use cases • CPU-bound • Simple inputs ◦ JSON ◦ Filenames ◦ data tables • Avoid chatty APIs (boundary cost)
58. Measure # of words in Shakespeare Ruby: 3 seconds Rust example: 30 ms
59. Good use cases ruby! { class WordCount { def search(path: String, search: String) -> i32 { let mut file = File::open(path).expect("could not open file."); // … lines.into_par_iter(). .map(|line| wc_line(line, search)) .sum() } } }
60. Good use cases • Use Rust libraries • Leverage Rust web browser tech • Mailer, Background job, Action Cable
61. ruby! { class InlineCSS { def inline(html: String, css: String) -> String { let doc = kuchiki::parse_html().one(html); let mut parser = parse_css::CSSParser::new(css); let rules = ...; for rule in rules { let matching_elements = ...; for matching_element in matching_elements { // insert style attribute } } // serialize } } } Good use cases
62. Roadmap • Greenfield project • Drop-in replacement • Reopen class • Ship to production • Non-traditional use-cases • Performance parity with C • Misc features and QoL improvements
63. Let's keep writing the ruby we love without the fear we'll hit a wall.
64. usehelix.com Thank You @hone02