Rust at Sentry

7 years later. How it worked. Presented at EuroRust 2023.

Transcript

1. Rust at Sentry 7 Years Later Armin @mitsuhiko Ronacher
2. What’s happening?
3. Who am I • Armin Ronacher • @mitsuhiko • https://lucumr.pocoo.org/ • I love Open Source • Flask, Insta, Jinja2, MiniJinja, …
4. What’s Sentry • https://sentry.io/ • Error and Crash Monitoring • Application Performance Monitoring • Session Replays etc. • Open Source (*) • A Python Shop *: some is BUSL licensed with a 3 year Apache 2 cliff
5. Errors and Crashes
6. Replays
7. Profiles
8. Traces
9. Why Rust? • Initially personal interest • Was really good for redistribution (sentry-cli) • Was really nice to expose to Python • Over time: we quite like it • Predictable at runtime • Tooling is really good
10. A Company’s Origin Story is a Legend • Memory gets foggy over time • Technology choices are less well informed and more incidental • Is Jane Street really successful because of OCaml?
11. Rust @ Sentry Stats • rust libraries + services: 180kLOC • Sentry Python Monolith: 455kLOC • Sentry TypeScript SPA: 612kLOC Third most popular language by LOC
12. Why we picked it
13. Predictable Runtime Behavior • Feels like Python • No whacky memory behavior ◦ (aside from suffering of fragmentation — hi jemallocator) • CPU usage mostly stays predictable • Performs well for a long time
14. Fits into Python • Great at extension modules • For us: cffi + milksnake (do not use!) • Nowadays: PyO3 + maturin
15. Unexpected Wins
16. Rust is Outbound • We quite actively contribute to external crates in Rust • We rarely do so in Python • Fork and depend on fork works well! • Cargo as tooling changes behavior
17. Standardized Tooling • One code style • Almost universally embraced lints • Rather well established patterns • Jumping between code-bases feels natural • Moving code between crates is trivial • Painless compiler upgrades
18. The DX is Dope • cargo • rustup • rust-analyzer • docs (std + crate)
19. Types and Borrow Checker • Modern Rust makes you a better programmer • Types for the most part are helpful • Borrow checker is not too annoying any more • Makes you suspicious of a lot of Python code
20. Unexpected Issues
21. Why is there so much memmove? • Large error types • String::clone and friends
22. Large Result Types (Large Errors) • The compiler sometimes is bad at optimizing result mapping
23. Shlemiel the Painter • Work gets progressively harder • Classic case: cstrings (strcat) • But also OFFSET + LIMIT in SQL Rust has a family of performance issues that are related • Fear of lifetimes cause bad lookups • String assigns become string clones
24. Shlemiel Paints the Entire Street For Every Dot • Add an offset to N tokens, clone entire source for every token
25. Strings are … not optimal • Maybe we should use more Arc<str>? • But Arc<str> is not particularly efficient • String’s extra capacity is odd in public APIs • Similar issue with Vec<u8> (broadcast to N sockets)
26. WTB
27. Errors • Still no stack trace on std::error::Error • Errors don’t have names (parsing Debug output)
28. Life Before Main / Registry • We would love a supported #[ctor] • Or a way to register startup functions
29. Async and Tokio
30. From Actix to Running our own Show • Started out with actix + actix-web • Actor frameworks feel great • Backpressure management is a giant pain and messy • Moved from pre-tokio 1.0 to async/await
31. How I learned to love the async Bomb • Use less async • Use More Channels • Embrace Backpressure • (Cancellations are still hard)
32. Rust is Good For Us
33. Rust Community: Let’s talk
34. Some Thoughts • Nobody is perfect • Building things is hard • Good intentions can still result in bad outcomes • Rust made it this far, let’s work on it together • We all are more nuanced in Person