EventSourcing.Live 2021 - Event system evolution - A Scientific Study on Event Sourcing, Lessons from Industry

Based on interviews with 25 engineers, in which we discussed 19 event sourced systems, we explore the rationale for and the context of event sourcing. Why do people use event sourcing, in what kind of systems do they apply it, and how satisfied are they. We also identify five challenges that practitioners experience: event system evolution, the steep learning curve, lack of available technology, rebuilding projections, and data privacy. We dig deeper into the first challenge, event system evolution. We uncover five tactics that engineers use in upgrading their event sourced system: versioned events, weak schema, upcasting, in-place transformation, and copy-and-transform. In this talk we share the results of our scientific study on event sourcing. We do not share the experience of one company, or one product, but the accumulated experiences of 25 engineers.

Transcript

1. Event system evolution A Scientific Study on Event Sourcing, Lessons from Industry
2. Michiel Lead Software Architect @ AFAS Software PhD candidate @ University Utrecht michiel.overeem@afas.nl @michielovereem https://movereem.nl Overeem
3. 25 engineers with 103 years of combined experience
4. RQ2 RQ3 RQ1 What types of systems apply event sourcing and why? How can event sourced systems be defined? What are the challenges faced in applying event sourcing?
5. THE PROBLEM
6. Project administration Marketing automation Website building Payment processing Content management Classified advertising …
11. Rationale Flexibility Complexity Trending Audit Immutability Strict Cut-off Mutable Technology .NET JVM PHP Ruby,Scala,Go
12. DDD Yes No MSA Yes No CQRS Yes No
13. THE SOLUTION
14. “business analysts are telling us what the events should be.” (E5) “you capture business changes as a flow of events, you align these events with real world events.” (E11)
15. internal versus external events
16. state propagation events
17. A single transaction crossing event & query database
18. THE CONSEQUENCES
19. Steep learning curve Learning curve Finding your way
20. Missing tools Related to the learning curve? Or the trend?
21. Projections Things might take a while
22. User Privacy Can we forget things?
23. Evolution Small steps, but how to stay stable?
24. Versioned Events Weak Schema Upcasters In-Place Copy- Transform 14/24 5/24 12/24 11/24 2/24
25. Michiel Overeem, Marten Spoor, Slinger Jansen, and Sjaak Brinkkemper. (2021). An Empirical Characterization of Event Sourced Systems and Their Schema Evolution - Lessons from Industry. In Journal of Systems and Software In Practice volume 178 (2021), 110970. https://dx.doi.org/10.1016/j.jss.2021.110970
26. Michiel michiel.overeem@afas.nl @michielovereem https://movereem.nl Overeem Event sourcing is a pattern that solves the three problems that modern systems face: flexibility, complexity, and reliability. These benefits give enough reason to incorporate event sourcing in modern systems.