Upgrade to Pro — share decks privately, control downloads, hide ads and more …

RustFest 2017 - Taking Rust To Production: Less...

RustFest 2017 - Taking Rust To Production: Lessons Learned From The Habitat Project

The Rust language is more than ready for production–it excels at it! The Habitat project team has been using Rust for a year in public and an extra year in stealth. With over 40,000 lines of Rust code so far, we maintain multiple CLI applications, a cross-platform process supervisor, a gossip subsystem, and a micro service-oriented distributed build system.

In this talk we will cover some of the challenges that Rust is uniquely qualified to address including static compilation, inlining dependencies, using clap for CLI parsing, pragmatically using native C libraries, maintaining a common codebase for Linux, Mac, and Windows, and on-boarding new Rustaceans. Have no fear: there has never been a better time to use Rust for your next service, application, or tool!

https://github.com/fnichol/talk-rustfest2017-habitat

Fletcher Nichol

June 14, 2017
Tweet

More Decks by Fletcher Nichol

Other Decks in Technology

Transcript

  1. Taking Rust to Production: Lessons Learned From the Habitat Project

    Fletcher Nichol RustFest 2017 April 30, 2017 Kyiv, Ukraine
  2. Fletcher Nichol Twitter: GitHub: Rust Users: Works on Habitat. Likes

    Rust. And summertime. Also, drums. @fnichol @fnichol @fnichol
  3. is a runtime system for your apps or other services

    provides realtime service configuration packages software with a deterministic build system loves your apps is primarily written in Rust Habitat
  4. Habitat Components in Rust The Supervisor The hab CLI A

    distributed build system (Builder)
  5. It's a growing codebase 01 > date 02 Fri 28

    Apr 2017 10:48:21 EEST 03 04 > cd habitat/ 05 06 > find . -name '*.rs' -exec cat {} \; \ 07 | egrep -v '^\s*$' \ 08 | egrep -v '^\s*//' \ 09 | wc -l 10 78551
  6. External libraries libarchive Archive reading and streaming libsodium All crypto

    applications (minus client TLS) zeromq rumor propagation, server communication
  7. Dynamic binary: file 01 > file target/x86_64-unknown-linux-gnu/release/hab 02 target/x86_64-unknown-linux-gnu/release/hab: ELF

    64-bit LSB » 03 shared object, x86-64, version 1 (SYSV), » 04 dynamically linked, » 05 interpreter /lib64/ld-linux-x86-64.so.2, » 06 for GNU/Linux 2.6.32, » 07 not stripped
  8. Dynamic binary: ldd 01 > ldd target/x86_64-unknown-linux-gnu/release/hab 02 linux-vdso.so.1 (0x00007ffd2c66b000)

    03 libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f6d524c4000) 04 libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f6d52071000) 05 libsodium.so.18 => /lib/x86_64-linux-gnu/libsodium.so.18 (0x00007f6d51db9000) 06 libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6d51118000) 07 /lib64/ld-linux-x86-64.so.2 (0x00007f6d52735000) 08 libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6d50e1a000) 09 libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f6d50bff000) 10 liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f6d509cc000) 11 libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f6d507bc000) 12 ...
  9. musl musl, a new standard library to power a new

    generation of Linux-based devices. musl is lightweight, fast, simple, free, and strives to be correct in the sense of standards-conformance and safety. - https://www.musl-libc.org/
  10. Build with musl: setup rustup target add x86_64-unknown-linux-musl # compile

    or install a musl package # containing library and headers
  11. Statically build hab program export LIBARCHIVE_LIB_DIR=/usr/local/lib export LIBARCHIVE_INCLUDE_DIR=/usr/local/include export LIBARCHIVE_LDFLAGS="-L/usr/local/lib

    -lz -llzma ..." export LIBARCHIVE_STATIC=true export OPENSSL_LIB_DIR=/usr/local/lib export OPENSSL_INCLUDE_DIR=/usr/local/include export OPENSSL_STATIC=true export SODIUM_LIB_DIR=/usr/local/lib export SODIUM_STATIC=true cargo build --release --target=x86_64-unknown-linux-musl
  12. Static binary: no external dependencies! 01 > file target/x86_64-unknown-linux-musl/release/hab 02

    target/x86_64-unknown-linux-musl/hab: ELF 64-bit LSB » 03 executable, x86-64, version 1 (GNU/Linux), » 04 statically linked, stripped 05 06 07 > ldd target/x86_64-unknown-linux-musl/release/hab 08 not a dynamic executable 09 10 11 > du -h target/x86_64-unknown-linux-musl/release/hab 12 7.6M target/x86_64-unknown-linux-musl/release/hab
  13. Habitat components Directory . ├── components │ ├── backline │

    ├── builder-admin │ ├── builder-admin-proxy ... │ ├── butterfly │ ├── butterfly-test │ ├── common │ ├── core ... │ ├── ruby-client │ ├── studio │ ├── sup │ └── win-users
  14. Cargo workspaces A workspace is a set of packages that

    will all share the same Cargo.lock and output directory. - The Rust Programming Language, 2nd Ed.
  15. Cargo.toml [workspace] members = [ "components/builder-admin", "components/builder-api", "components/builder-core", ... "components/butterfly",

    "components/butterfly-test", "components/common", ... "components/hab", "components/http-client", "components/net", "components/sup", ]
  16. components/hab/Cargo.toml 01 [package] 02 name = "hab" 03 version =

    "0.0.0" 04 build = "../build.rs" 05 workspace = "../../"
  17. components/build.rs (1/2) 01 fn main() { 02 let version =

    match env::var("PLAN_VERSION") { 03 Ok(ver) => ver, 04 _ => read_version(), 05 }; 06 let mut f = File::create( 07 Path::new(&env::var("OUT_DIR").unwrap()) 08 .join("VERSION")).unwrap(); 09 f.write_all(version.trim().as_bytes()).unwrap(); 10 }
  18. components/build.rs (2/2) 01 fn read_version() -> String { 02 let

    ver_file = Path::new( 03 &env::var("CARGO_MANIFEST_DIR").unwrap()) 04 .parent() 05 .unwrap() 06 .parent() 07 .unwrap() 08 .join("VERSION"); 09 let f = File::open(ver_file).unwrap(); 10 let mut reader = BufReader::new(f); 11 let mut ver = String::new(); 12 reader.read_line(&mut ver).unwrap(); 13 ver 14 }
  19. components/sup/Cargo.toml [dependencies] ansi_term = "*" bitflags = "*" byteorder =

    "*" clap = { version = "*", features = [ "suggestions", "color", "unstable" ] } env_logger = "*" features = "*" glob = "*" habitat_butterfly = { path = "../butterfly" } habitat_common = { path = "../common" } ...
  20. Cross platform tips Centralize conditional code into a few places

    Keep public API calls consistent, if possible Read the Rust codebase, it has great examples!
  21. components/core/src/os/filesyst em/mod.rs 01 #[cfg(windows)] 02 mod windows; 03 04 #[cfg(windows)]

    05 pub use self::windows::{chown, chmod, symlink}; 06 07 #[cfg(not(windows))] 08 mod linux; 09 10 #[cfg(not(windows))] 11 pub use self::linux::{chown, chmod, symlink};
  22. components/core/src/os/filesyst em/linux.rs 01 use libc::{self, c_int, c_char, mode_t}; 02 03

    pub use std::os::unix::fs::symlink; 04 use std::ffi::CString; 05 06 pub fn chown(path: &str, uid: u32, gid: u32) -> Result<c_int> { 07 // ... 08 } 09 10 pub fn chmod(path: &str, mode: u32) -> Result<c_int> { 11 // ... 12 }
  23. components/core/src/os/filesyst em/windows.rs 01 // ... 02 03 pub fn chown(path:

    &str, uid: String, gid: String) 04 -> Result<c_int> { 05 path_exists(path) 06 } 07 08 pub fn chmod(path: &str, mode: u32)-> Result<c_int> { 09 path_exists(path) 10 } 11 12 pub fn symlink<P: AsRef<Path>, Q: AsRef<Path>>(src: P, dst: Q) 13 -> io::Result<()> { 14 unimplemented!(); 15 }
  24. Onboarding tips Give new members time and space to learn,

    experiment, ask questions Pair on features Talk about architecture and code's "point of view" Talk about .unwrap() and .expect() Be patient--it gets really good if you put the time in!