Twenty Thousand Leagues Under Platform Engineering

A 3-year long story about getting an enterprise from "zero to hero" from manually deploying monoliths, to CI/CD, microservices, Gitops and building an IDP in 2020, before DevEx was cool.

This talk is a behind-the-scenes of an adventurous journey within a medium-sized enterprise, navigating through challenges and obstacles of going Cloud Native with a blend of diplomacy, acknowledging the complexity of convincing management, developers and stakeholders while leveraging various open-source tools to get from on-premise all-manual development to cloud automation and beyond.

Lifting a monolith, re-architecting apps to microservices, deploying to K8s, implementing helm, then all the way to integrating different pipelines with ArgoCD, FluxCD and JenkinsX without forgetting observability.

Transcript

1. KCD Austria 2024 PRESENTER Twenty Thousand Leagues Under Platform Engineering Jules Verne’s trip down the Platform Engineering Depths Matteo Bianchi Kubernetes Release Team - Comms Lead v1.32
2. Twe󰈝󰉄y-󰉃󰈋o󰉉s󰇽󰈝󰇶 Matteo Bianchi’s Le󰈀g󰉊e󰈻 U󰈞d󰈩󰈸 Pla󰉃󰇿󰈡r󰈚 E󰈞gi󰈝󰈩󰇵󰈹in󰈇
3. Abo󰉉󰉃 󰈛󰇵 @mbianchidev (pretty much anywhere) Cloud Native Consultant and Developer Advocate by day, Metalcore Singer by night Former startup CTO, Staff DevOps/SRE I love building (and breaking) platforms
4. Age󰈝󰇶󰈀 The end? Takeaways and Q&A (maybe) Introduction Values of a Platform A Platform Tale The journey of a big corp turning to Platform Engineering
5. The 󰈑󰈡󰉊󰈹ne󰉘 󰈼t󰈀󰈸󰉄s 󰉒󰈏󰉄h… - Manual deployments - Ancient monoliths - On-premise infrastructure - A real datacenter (in a single remote location)
6. An u󰈝󰇸󰈩r󰉃󰇽i󰈞 g󰈡󰇽󰈗 - Where are we headed, captain? - What is this platform? - What do we need to get there?
7. The 󰈰󰈀󰉊󰉃i󰈘󰉉s - Legacy, complex but powerful - In need of modernization - Full of bottlenecks - Difficult to steer and scale The initial crew is very experienced but stuck in the Old Ways.
8. Bef󰈡󰈸󰇵 󰈘a󰉉n󰇹󰈋󰈏n󰈇 Convincing developers and other tech people requires building a Developer Experience loop Top-Down? Bottom-up? Both? Convincing management to embark on the deep-platform dive requires Return on Investment (ROI) considerations. Doing both could seem an irrational choice, surely requires extensive resources, time included.
9. Pla󰉃󰇿󰈡r󰈚 RO󰈾 • Align to business goals ◦ Startup? Unclear PMF, still figuring GTM? Sorry, no platforms for you. • Measure business metrics before…and after ◦ It’s not only DORA, it could be anything else, depending on your end product • Survey and measure user satisfaction, create a Developer Experience culture around the platform
10. Fir󰈻󰉄 l󰈀󰉊󰈝󰇸h? Di󰈻󰈀󰈼t󰇵󰈸. - Heavy lifting of monolithic applications - Breaking down backend and frontend - then divide each in domains - then microservices How do we manage dependencies? Who knows what we need to do?
11. Sec󰈡󰈝󰇶 l󰇽u󰈝󰇸h? Gu󰈩󰈻󰈼. - Kubernetes is hella complex - Automating deployments it’s nice but… - No conventions - Tool sprawling (Jenkins X, Spinnaker, Flux, Argo, Helm…) - Operational challenges multiply How do we manage Kubernetes? What in the tool sprawl?!
12. Thi󰈸󰇶 t󰈎󰈚󰇵’󰈼 a c󰈊󰈀󰈹m… or 󰈻󰈡 󰉄h󰇵󰉘 󰈼ay! We went for a complete rework and standardization the interfaces for our CI/CD systems, completely and from scratch, product by product. Introducing new tools along the way! - Artifactory - GitLab CI - SonarQube
13. To 󰉂󰈎t 󰈢󰈸 󰉄o Op󰈻? We moved every single repo to GitLab so we could start leveraging GitOps with a single source of truth. We had to take into account the eternal dilemma ArgoCD vs FluxCD. Guess who won!
14. The 󰉒󰈡󰈹l󰇷 󰇿ro󰈚 󰈀 󰈦󰈢r󰉃󰈋ol󰈩 We need to harness our tools, captain! - New Relic - Brand new Telemetry team - Educate developer to instrument their code - Educate infra to install agents: everywhere
15. The 󰇹󰈹󰈩w Tools, processes, software… but what about people? Team Topology changes were needed to ensure the success of the Platform. We started creating new roles such as Platform Owners.
16. Al󰈚o󰈼t 󰉃󰈋󰈩r󰇵… The urge of adopting an IDP became stronger as our product teams grew demand of standardized interfaces for services. We considered different alternatives: Buy vs Build Build won at the beginning We started… and failed.
17. A pe󰈩󰈔 󰈏󰈞 t󰈊e 󰈀󰇻y󰈻󰈼 Create golden paths or the shortest (and thinnest) paths for your code to be shipped in production, decorate them after! Composability Standardization Build your platform API-first on a cloud agnostic infrastructure piece (e.g. Kubernetes) and build like a multiplayer game, supporting multiple plug-ins from day 0
18. The 󰈑󰈡󰉊󰈹ne󰉘 󰈩󰈞d󰈻 󰉓󰈏t󰈊… - Automated GitOps deployments - A self service platform - Public Cloud widely adopted - Developer Experience
19. Les󰈻󰈡󰈞s 󰈗󰇵a󰈹n󰈩󰇷 Tools are fungible, you can change them, you will change them, the important thing is creating a Platform Engineering culture. Fail small People > Tools Business buy-in Remember fail fast? It’s still valid, but also… fail small! Do that PoC/PoV and don’t be afraid of admitting you were wrong. No money, no game! You need management to be bought on your platform or it’s not going anywhere. Business matters.
20. Las󰉃 󰇻󰉉t 󰈝󰈢󰉄 le󰈀󰈻󰉄 kubetrain.io
21. Cre󰇷󰈎󰉄s https://tag-app-delivery.cncf.io/wgs/platforms/
22. Tha󰈝󰈕 y󰈡󰉊 󰇾o󰈹 󰈀󰉅󰇵n󰇷i󰈞g! @mbianchidev p.s. I’m open to work!
23. Res󰈡󰉊󰈸󰇸e Pag󰈩 Titles: Caveat Brush Headers: Lato Body Copy: Lato You can find these fonts online too. Happy designing! Use these design resources in your Canva Presentation.