An ARM and a leg: Migrating to Graviton

(Given at AWS & DevOps TLVCommunity Fest: Byte-Sized DevOps for Startups 2024)

With Android dominating the mobile landscape, Apple Silicon disrupting the market and AI bringing new opportunities, the industry is seeing a seismic shift: the long-lived hegemony of Intel’s x86 architecture is suddenly and at long last under real threat. With vendors introducing offerings at an alarming rate and organizations scrambling to adapt, there has never been a better time to consider arm64.

Having recently completed a large-scale migration to AWS’s Graviton processors, this is an opportune time to share what we’ve learned. What does adopting a new architecture entail? How does an organization get started, what are the technical risks, the how’s and the why’s? Join me on a technical adventure for fun and profit!

Transcript

1. An ARM and a Leg Or: How I learned to stop worrying and love Graviton T o m e r G a b e l Tel-Aviv, 29 May 2024 //
2. First things first 1. This isn’t a sales pitch - I don’t work for Amazon - I don’t know your situation Photo: Money by 401(K) 2012 (CC)
3. First things first 1. This isn’t a sales pitch - I don’t work for Amazon - I don’t know your situation 2. This isn’t really about Graviton - arm64 is all the rage - Many options out there
4. Second things second 1. Hi, I’m Tomer Gabel! - Engineer, architect, grump - Freelancer & consultant
5. Second things second 1. Hi, I’m Tomer Gabel! - Engineer, architect, grump - Freelancer & consultant 2. Helped a large client migrate to Graviton
6. Second things second 1. Hi, I’m Tomer Gabel! - Engineer, architect, grump - Freelancer & consultant 2. Helped a large client migrate to Graviton 3. My opinions are my own
7. Let’s rock & roll
8. Why bother? 1. arm64 is abuzz but isn’t new - Old hat for embedded software - Virtual monopoly in mobile - New to desktop- and server-class Photos: Raimond Spekking , Wandelopa, Skitterphoto (CC)
9. Why bother? 2. arm64-based servers promise better value for money - Forter runs thousands of nodes - Cost savings aggregate quickly Photo: Stack of coins by Jam Willem Doormembal (CC)
10. Why bother? 3. Main development environment (MacOS) is now on arm64, requiring: - arm64 builds to work locally - arm64 on server to debug effectively Photo: M2 Macbook Air Starlight model by KKPCW (CC)
11. Graviton at “We build systems to protect eCommerce from fraud and abuse. We take pride in building the foundations for a safer Internet at massive scale.” --forter.dev
12. “We build systems to protect eCommerce from fraud and abuse. We take pride in building the foundations for a safer Internet at massive scale.” --forter.dev Graviton at
13. eCommerce safer at scale • High reliability, low latency • Security reigns supreme • Everything is auditable • Tightly regulated • Risk-averse environment Graviton at
14. 1. Heterogenous workloads - Directly on VMs in EC2 - Dockerized on EC2 - Containerized on EKS 2. Polyglot stack - Python, Node.js, JVM… Graviton at
15. This represents the worst-case scenario for migration.
16. The Two Towers Virtual Machines (EC2) Provisioning (Chef) Forter setup Dependencies Base image build (Packer) Initial setup Prerequisites Image source Ubuntu 22.04 / CIS-hardened Docker Containers Service layers App code Glue logic Forter base images Customization “Blessed” stacks Image source ubuntu:22.04 alpine:3.8
17. The first milestone Bring-up • Deployment infrastructure • Compatible base image (Packer) Provision • Chef + Ruby gems • Base recipes, components Serve • Bootstrap base images • Update components as needed
18. arm64 support in Linux is old hat • But the ecosystem… ugh • Trouble vectors include: - Docker <= 19.x - Chef cookbooks (docker, lvm) - Vagrant + AWS provider - Python 2.x broken on Ubuntu! - No Node.js <14 builds There’s a pattern here… Photo: Gorilla Scratching Head by Eric Kilby (CC)
19. Extending the build system 1. Custom build system - Jenkins + Pipeline + plugins 2. Self-hosted runners - Same base images - Same provisioning flow - Same deployment infrastructure
20. Bootstrapping 1. Emulation does work! - qemu - binfmt
21. Bootstrapping 1. Emulation does work! - qemu - binfmt 2. Well, kind of… - Docker version - Bugs all the way down
22. Extending the build system
23. 1. Bootstrapped build system with x64/arm64 native runners 2. Full stack of arm64 Docker base images 3. Modified Jenkinsfile with support for multiple architectures
24. Will it blend? “… cluster is behaving well with read latency of P95=P99=1ms …” vs. “~20% decrease in supported RPS"
25. Will it blend? “… cluster is behaving well with read latency of P95=P99=1ms …” vs. “~20% decrease in supported RPS" It depends. You knew this was coming.
26. arm64 migration: Key Takeaways 1. Migrating is easier than you think, although: - Homegrown systems may require delicate surgery - It forces you to pay technical debt 2. The most useful advice bar none: - Use emulation for bootstrapping only - Use uname –m and docker buildx imagetools
27. tomer@substrate.co.il @substrate_eng https://github.com/holograph Thank you for listening Questions? Substrate