ABS2024: Navigating the Azure Cloud at Digitec Galaxus by Olivier Girard & Gerald Schermann

Our story starts in 2015 with our on-prem .NET monolith running on Windows Server, accompanied by a single, clunky MS SQL database, powered by fusion-io disks and many other touchable hardware in a server-room - all built-up with about 5 teams back then. Today, our system is highly distributed and modular, entirely operating in the cloud, and processing significantly more data. It is continuously developed and maintained by over 45 Scrum teams. In our talk, we will delve into our experiences spanning the last 9 years – discussing how we work and organize ourselves, tracing the evolution of our system architecture up to the present, and, of course, exploring the role Azure plays in it all. Spoiler: not everything has always gone according to plan…
🙂 OLIVIER GIRARD ⚡️ Domain Architect for Platform and Online Shop @ Digitec Galaxus
🙂 GERALD SCHERMANN ⚡️ Software Architec @ Digitec Galaxus

Transcript

1. Navigating the Azure Cloud at Digitec Galaxus Our Journey and Lessons Learned Digitec Galaxus AG Olivier Girard | Gerald Schermann 16.05.2024
2. Olivier Girard Domain Architect Platform / Online Shop Gerald Schermann Software Architect Platform & SysOps “I paint boxes” “I build stuff for other devs”
3. Our Cloud Journey 1 - The Past 2 - The Present 3 – The Learnings 0 - About Us
4. What we're about and how we work
5. Innovative Ambitious Responsible Cooperative Piratesque
6. We also run six additional shops in the EU galaxus.ch digitec.ch galaxus.fr galaxus.de galaxus.at galaxus.it galaxus.nl galaxus.be
7. Our Applications
8. 8 Our Cloud Journey – The Past
9. On-prem Hosting Manual Deployment 2015 … digitec.ch/galaxus.ch Revenue 645M Product Catalog 300K Orders 2M 350 300 250 200 150 100 50 2007 2015 2018 2024 Employees Product Development Monolith Layered Architecture ERP OnlineShop DWH Monolith DB TL PO SA Dev 5 Scrum Teams 2015
10. Cloud Hosting (Azure/GCP) Continuous Delivery ... 2018 Monolith Layered Architecture ERP OnlineShop Logistics DWH Monolith DB CH + galaxus.de Revenue 970M Product Catalog 2M Orders >4M 350 300 250 200 150 100 50 2007 2015 2018 2024 Employees Product Development TL PO SA Dev > 10 Scrum Teams OnlineShop vNext GraphQL Community Search Microservices 2018
11. Hosting pre-2017 CDN Windows LB VMs Shop/ERP VMs Shop/ERP VMs Shop/ERP VMs Our own hardware … 2017
12. Hosting pre-2017: lift & shift CDN Windows LB VMs Shop/ERP VMs Shop/ERP VMs Shop/ERP VMs Microservices in AKS IIS reverse proxy … 2017
13. Hosting in 2017 CDN Windows LB VMs Shop/ERP VMs Shop/ERP VMs Shop/ERP VMs Microservices in AKS IIS reverse proxy 2017 BLACK FRIDAY
14. Black Friday 2017 2017
15. Hosting in 2017 CDN Windows LB VMs Shop/ERP VMs Shop/ERP VMs Shop/ERP VMs Microservices in AKS IIS reverse proxy BLACK FRIDAY 2017 Redis
16. Hosting in 2018 CDN Windows LB VMs Shop/ERP VMs Shop/ERP VMs Shop/ERP VMs Microservices in AKS Routing Layer AKS in-place upgrades? IIS reverse proxy Microservices in AKS Redis 2018
17. Hosting in 2018 CDN Windows LB VMs Shop/ERP VMs Shop/ERP VMs Shop/ERP VMs Routing Layer IIS reverse proxy Microservices in AKS Redis 2018 Single point of failure? Maybe? Wait until 2022 … the day the routers died
18. Hosting in 2019 / 2020 CDN Shop/ERP VMs Shop/ERP VMs Shop/ERP VMs Routing Layer Microservices in AKS Redis ERP VMs Containerization facts: From .NET Full Framework to .NET Core Replacing all Windows-only dependencies 9 months 2020
19. DevOps – Early Days VCS & Continuous Delivery SRE Team No deployments on Friday! Logging / Monitoring / Alerting Application Insights Prometheus Logs, Alerts, Metrics K8s (Workload) Metrics, Alerts Metric correlation & sampling Scaling issues Grafana 2020
20. More and more Data, and distributed DWH Monolith DB MS SQL (> 2 TB) ShopSearch Elastic ERP OnlineShop Business Intelligence Community ShopProductCatalog Mongo DB Data propagation over Azure Service Bus nightly full-copy 2020
21. Our Cloud Journey – The Present
22. Containers everywhere … 2024 ERP Data Platform Galaxus in 7 countries Revenue 2370M Product Catalog 6M Orders 15M 350 300 250 200 150 100 50 2007 2015 2018 2024 Employees Product Development 35M Sessions/month 200 Page Views/sec >700 Services/DBs in prod >6M LoC TL PO SA Dev > 45 Scrum Teams Online Shop ShopProduct Modular Monolith Microservices (or Microliths) Unstructured Monolith Layered Architecture … Monolith DB 2024
23. Modular Architecture E-Commerce by DG Customer Service Retail Editorial HR … Sales Order Fullfillment Finance Marketplace Sales Category Management Warehousing Purchasing Provider Management Community Marketing Community
24. Data Platform Modular Architecture ERP Online Shop Backend Data Analysis Implementation 2024
25. Modular Architecture Monolith Online Shop Frontend 2024 Hosting/Ops DEV OPS *
26. Hosting CDN Routing Layer Microservices in AKS Redis CPU-bound Memory-bound «No interruptions» «Quarantine» Node Pools + Spot Instances Scale to Zero + Previews Infrastructure as Code Multi-Cluster Management 2024
27. Preview Deployments with Knative 2024 Preview Pod per PR “Regular” Pods based on main Other Services Service calls
28. Preview Deployments with Knative 2024
29. The day the routers died reboot recreation 2022
30. The day the routers died reboot recreation 2022
31. Culture: Post Mortems Document Incident Understand Root Cause Preventive Actions 2024
32. Data (is gold) event-carried state transfer 2024 single writer principle data on the inside data on the outside
33. Data Platform Big Query Data (is gold) event-carried state transfer 2024 single writer principle data on the inside data on the outside data mesh
34. Autonomous Teams TL Scrum Team PO SA Dev mission OKR x-functional autonomous customers business 6-10 members 2024 DEV OPS *
35. We do Dev*Ops
36. Internal Developer Platform TL Feature Team PO SA Dev Platform Database Hosting Dev Tooling Observability Messaging / Kafka Data Platform Delivery (CI/CD) Networking Security TL Platform Team PO Platform Architect DevOps Engineer Data Engineer Data Team PO Data Scientist TL support / enabling / governance customer needs / feedback 2024
37. Dev*Ops and Platform Observability Datadog VCS & Continuous Deployment Security Kyverno Network Policies Managed Identities Security Team 2024
38. Developer Tools Dev Tools & Local Development Developer Portal DB Self-Service & (Cost) Transparency Missing Pieces Feature Team 2024
39. 2024 Developer Tools Dev Tools & Local Development Developer Portal DB Self-Service & (Cost) Transparency Missing Pieces Feature Team
40. 2024 Developer Tools Dev Tools & Local Development Developer Portal DB Self-Service & (Cost) Transparency Missing Pieces Feature Team
41. Developer Tools Dev Tools & Local Development Developer Portal DB Self-Service & (Cost) Transparency Missing Pieces Feature Team 2024
42. Our Cloud Journey – The Learnings
43. Top 3 Lessons Learned not-invented-here vs. 3rd party / managed 1 distributed vs. modular 2 one-size fits all vs. specific solution 3 …
44. Next Up in Our Journey de-couple to scale (sagas, streaming) event-driven architecture Simplify and more self-service to increase productivity <DX> system overview / observability, explicit know how and governance holistic system view service mesh, federated governance and security multi/hybrid cloud …
46. galaxus.ch/jobs Olivier Girard Domain Architect – Online Shop / Platform https://www.linkedin.com/in/oligirard/ Gerald Schermann Software Architect – Platform https://www.linkedin.com/in/gerald-schermann/ Tech Blog