Upgrade to Pro — share decks privately, control downloads, hide ads and more …

.NET Day 2024: Curious Code and Where to Find it

dotnetday
September 11, 2024

.NET Day 2024: Curious Code and Where to Find it

More info at: https://dotnetday.ch

dotnetday

September 11, 2024
Tweet

More Decks by dotnetday

Other Decks in Technology

Transcript

  1. an opinionated talk about my curious experience with a microservice

    project 05.09.2024 WE KNOW HOW. 4 your mileage may vary!
  2. Alexander Kayed 05.09.2024 WE KNOW HOW. 5 Alex MSc ETH

    in Computer Science github.com/akade linkedin.com/alexander-kayed Senior Software Engineer / Architect / Trainer / Speaker
  3. Alexander Kayed 05.09.2024 WE KNOW HOW. 6 Alex MSc ETH

    in Computer Science github.com/akade linkedin.com/alexander-kayed Senior Software Engineer / Architect / Trainer / Speaker
  4. Alexander Kayed 05.09.2024 WE KNOW HOW. 7 Alex MSc ETH

    in Computer Science github.com/akade linkedin.com/alexander-kayed Senior Software Engineer / Architect / Trainer / Speaker
  5. Alexander Kayed 05.09.2024 WE KNOW HOW. 8 Alex MSc ETH

    in Computer Science github.com/akade linkedin.com/alexander-kayed Senior Software Engineer / Architect / Trainer / Speaker
  6. The Project 05.09.2024 WE KNOW HOW. 10 • SaaS /

    Multi-Tenant • Diverse clients • Based all over Switzerland
  7. The Project 05.09.2024 WE KNOW HOW. 11 • Long-running inhouse

    project • Since fall 2017 • tech lead 2022 – 2024 • 3-5 engineers • Agile
  8. The Project 05.09.2024 WE KNOW HOW. 12 • Long-running inhouse

    project • Since fall 2017 • tech lead 2022 – 2024 • 3-5 engineers • Agile • «Kanbanesque»
  9. The Project 05.09.2024 WE KNOW HOW. 13 • Long-running inhouse

    project • Since fall 2017 • tech lead 2022 – 2024 • 3-5 engineers • Agile • «Kanbanesque» • Regular exchange with «working group»
  10. The Project 05.09.2024 WE KNOW HOW. 14 • Long-running inhouse

    project • Since fall 2017 • tech lead 2022 – 2024 • 3-5 engineers • Agile • «Kanbanesque» • Regular exchange with «working group» • Happy clients and growing fast!
  11. The Project 05.09.2024 WE KNOW HOW. 15 • Long-running inhouse

    project • Since fall 2017 • tech lead 2022 – 2024 • 3-5 engineers • Agile • «Kanbanesque» • Regular exchange with «working group» • Happy clients and growing fast! Public Domain – Battle of Dornach
  12. The Project 05.09.2024 WE KNOW HOW. 16 • Microservices •

    CQRS – Command Query Separation Principle • Event Sourcing
  13. The Project 05.09.2024 WE KNOW HOW. 17 • Microservices •

    CQRS – Command Query Separation Principle • Event Sourcing ~per aggregate
  14. The Project 05.09.2024 WE KNOW HOW. 18 • Microservices •

    CQRS – Command Query Separation Principle • Event Sourcing Separated read / write service ~per aggregate
  15. The Project 05.09.2024 WE KNOW HOW. 19 • Microservices •

    CQRS – Command Query Separation Principle • Event Sourcing Separated read / write service ~per aggregate All aggregates are materialized in-memory («hydrated»)
  16. The Project 05.09.2024 WE KNOW HOW. 20 • Microservices •

    CQRS – Command Query Separation Principle • Event Sourcing Separated read / write service ~per aggregate No Codesharing Multirepo All aggregates are materialized in-memory («hydrated»)
  17. All aggregates are materialized in-memory («hydrated») The Project 05.09.2024 WE

    KNOW HOW. 21 • Microservices • CQRS – Command Query Separation Principle • Event Sourcing Separated read / write service ~per aggregate No codesharing
  18. The Project 05.09.2024 WE KNOW HOW. 26 Dev loop: Manual

    running of pipelines and manual updating of packages
  19. The Project 05.09.2024 WE KNOW HOW. 28 To Release while

    (something changes) Update all packages & push Wait for CI-Builds to complete
  20. Motivation Microservices 05.09.2024 WE KNOW HOW. 36 • Changeability •

    Availability • Strong module boundaries • Independent deployment • Independent scalability • Independent technology choice • Microservices are very small • No stable contracts • Tight coupling • Distributed tight coupling!! • Lackluster tooling support over multiple repos and nuget packages • Heterogeneity of services • Anticipating consequences is hard • Curious day-to-day development
  21. Motivation Microservices 9/5/2024 WE KNOW HOW. 37 • Changeability •

    Availability • Strong module boundaries • Independent deployment • Independent scalability • Independent technology choice • Microservices are very small • No stable contracts • Tight coupling • Distributed tight coupling!! • Lackluster tooling support over multiple repos and nuget packages • Heterogeneity of services • Anticipating consequences is hard • Curious day-to-day development
  22. Motivation Microservices 9/5/2024 WE KNOW HOW. 38 • Changeability •

    Availability • Strong module boundaries • Independent deployment • Independent scalability • Independent technology choice • Target: 99.999% • = ~5 min/y! • Rehydration of in-memory state (Event Sourcing) takes 2-3h • Clients work only 9 to 5
  23. Motivation Microservices 05.09.2024 WE KNOW HOW. 39 • Changeability •

    Availability • Strong module boundaries • Independent deployment • Independent scalability • Independent technology choice • Target: 99.999% • = ~5 min/y! • Rehydration of in-memory state (Event Sourcing) takes 2-3h • Clients work only 9 to 5
  24. Motivation Microservices 05.09.2024 WE KNOW HOW. 40 • Changeability •

    Availability • Strong module boundaries • Independent deployment • Independent scalability • Independent technology choice • Theoretically possible • But never used! • Different library versions in different services • => hard to debug / know, what is used where • => possibility of inconsistent data (in-memory state)
  25. Motivation Microservices 05.09.2024 WE KNOW HOW. 41 • Changeability •

    Availability • Strong module boundaries • Independent deployment • Independent scalability • Independent technology choice • Theoretically possible • But never used! • Different library versions in different services • => hard to debug / know, what is used where • => possibility of inconsistent data (in-memory state)
  26. Motivation Microservices 05.09.2024 WE KNOW HOW. 42 • Changeability •

    Availability • Strong module boundaries • Independent deployment • Independent scalability • Independent technology choice • Rehydration time • Dynamic, horizontal scaling not possible / slow • Concurrent Users: max. 1000 • Performance / Latency • In-Memory but often full-scans • Network Hops
  27. Motivation Microservices 05.09.2024 WE KNOW HOW. 43 • Changeability •

    Availability • Strong module boundaries • Independent deployment • Independent scalability • Independent technology choice • Rehydration time • Dynamic, horizontal scaling not possible / slow • Concurrent Users: max. 1000 • Performance / Latency • In-Memory but often full-scans • Network Hops
  28. Motivation Microservices 05.09.2024 WE KNOW HOW. 44 • Changeability •

    Availability • Strong module boundaries • Independent deployment • Independent scalability • Independent technology choice • All «own» services are .NET • Heterogeneity is more a disadvantage • But: • ElasticSearch • Kibana • Prometheus • Grafana • PowerShell Container • …
  29. Motivation Microservices 05.09.2024 WE KNOW HOW. 45 • Changeability •

    Availability • Strong module boundaries • Independent deployment • Independent scalability • Independent technology choice • Traceability (Event Sourcing)
  30. Lessons learned 05.09.2024 WE KNOW HOW. 46 • Concepts have

    trade-offs and effect each other • Data persistence (event sourcing) & queries affect each other • Queries over all aggregates  dynamic rehydration impossible • Always ask why • Horizontal Scaling • Fast responses during peak traffic cost-effectivity during low traffic
  31. Curious Code – an example 05.09.2024 WE KNOW HOW. 48

    What id format should we use? distributed system
  32. Curious Code – an example 05.09.2024 WE KNOW HOW. 49

    What id format should we use? distributed system database friendly
  33. Curious Code – an example 05.09.2024 WE KNOW HOW. 50

    What id format should we use? distributed system database friendly Twitter Snowflake (64bit) 41 bit timestamp Machine id 10 bits 12 bits sequence
  34. Curious Code – an example 05.09.2024 WE KNOW HOW. 51

    What id format should we use? distributed system database friendly Twitter Snowflake (52bit) 41 bit timestamp Machine id 10 bits 12 bits sequence No integers in JS-Land
  35. Curious Code – an example 05.09.2024 WE KNOW HOW. 52

    What id format should we use? distributed system database friendly Twitter Snowflake (52bit) 41 bit timestamp 4 bits 6 bits No integers in JS-Land
  36. Curious Code – an example 05.09.2024 WE KNOW HOW. 53

    52-bit snowflake ids Frontend needs offline support for new ids
  37. Curious Code – an example 05.09.2024 WE KNOW HOW. 55

    ID Prefetching 52-bit snowflake ids
  38. Curious Code – an example 05.09.2024 WE KNOW HOW. 56

    What id format should we use? distributed system database friendly Twitter Snowflake (52bit) 41 bit timestamp 4 bits 6 bits Guid? What indices would suffer? Max 16 services? Max 64 Ids/ms 52-bit snowflake ids
  39. Lessons learned 05.09.2024 WE KNOW HOW. 58 • There are

    always dependencies! HTTP Container
  40. Lessons learned 05.09.2024 WE KNOW HOW. 59 • There are

    always dependencies! Method call on interface Class
  41. Martin Fowler 05.09.2024 WE KNOW HOW. 61 1. Almost all

    the successful microservice stories have started with a monolith that got too big and was broken up.
  42. Martin Fowler 05.09.2024 WE KNOW HOW. 62 1. Almost all

    the successful microservice stories have started with a monolith that got too big and was broken up. 2. Almost all the cases where I've heard of a system that was built as a microservice system from scratch, it has ended up in serious trouble. https://martinfowler.com/bliki/MonolithFirst.html
  43. Where are the people? 05.09.2024 WE KNOW HOW. 67 Refactoring

    Is Not Just Clickbait - Kevlin Henney - NDC Oslo 2022 “Software Architecture is for developers to live inside” Kevlin Henney
  44. Why Microservices? 05.09.2024 WE KNOW HOW. 68 • […] One

    of its goals is that teams can develop and deploy their services independently of others. […] Quote from https://en.wikipedia.org/wiki/Microservices
  45. Why Microservices? 05.09.2024 WE KNOW HOW. 69 • […] One

    of its goals is that teams can develop and deploy their services independently of others. […] Spotify: >3000 Engineers A single team (~4 members) Quote from https://en.wikipedia.org/wiki/Microservices
  46. Improvements 05.09.2024 WE KNOW HOW. 70 • Code Sharing •

    Harmonize services • Merge tiny services • Automatization • Tracing & Logging
  47. Multi- to Monorepo 05.09.2024 WE KNOW HOW. 72 • Matches

    organization • One team • One application • One deployment unit • One PR per feature • Main challenge: Hotfix & merging during transition Conclusion • Game changer • Easier day-to-day work • Productivity gains!
  48. Nuget to project references 05.09.2024 WE KNOW HOW. 73 •

    Advantages project references • Tooling support • Direct builds & testing • Reasoning over changes and consequences much easier • Main challenge: complicated pipeline triggers
  49. Project references 05.09.2024 WE KNOW HOW. 74 Conclusion • Even

    bigger game changer • Immensely easier day-to-day live • Huge productivity gains! Together with the monorepo: Essential for moving foward with refactoring & features
  50. Automation via our own CLI 05.09.2024 WE KNOW HOW. 75

    • Generates & validates pipelines • Trigger always match project references • Minimal friction for refactoring or bigger updates (i.e. .NET 9) • … • Build & Deploy • Reduces day-to-day toil • Minimizing errors due to cumbersome manual steps
  51. CLI 05.09.2024 WE KNOW HOW. 76 Conclusion • Worthwhile •

    Less sources of errors • (Big) refactoring much easier
  52. Conclusion 05.09.2024 WE KNOW HOW. 77 • Never stop asking

    why • Be curious! • Be mindful with complexity • There is no one size fits all! • Do not forget the people including developers • Improved dev live is a win-win situation
  53. Conclusion 05.09.2024 WE KNOW HOW. 78 • Never stop asking

    why • Be curious! • Be mindful with complexity • There is no one size fits all! • Do not forget the people including developers • Improved dev live is a win-win situation • Interesting and complex but also very successful project. Keep calm and refactor!