Design insights from unit tests @ itkonekt 2023

If tests are hard to write, the production design is crappy - goes an old saying. Indeed, writing unit tests gives you one of the most comprehensive, yet brutal, feedback about the design of your production code, but if it comes too late, many developers can’t stand it anymore and they will either stop testing or test more superficially. At the other extreme, others struggle to write contrived, fragile tests full of mocks that end up frustrating more than helping them. This talk reviews the main hints that unit tests provide you, from the most obvious improvements to some of the most subtle design principles.

Transcript

1. Your tests are trying to tell you something ... 10 design hints you were missing ↓ Read the article: https://victorrentea.ro/blog/design-insights-from-unit-testing/ Ways Victor can help you: https://victorrentea.ro by Victor Rentea
2. 👉 victorrentea.ro/training-offer Hi, I'm Victor Rentea 🇷🇴 PhD(CS) Java Champion, 17 years of code, code, code, code, code.... Consultant & Trainer: 5000 developers of 100+ companies in EU: ❤ Clean Code, Architecture, Unit Tes3ng 🛠 Frameworks: Spring, Hibernate, WebFlux ⚡ Java Performance, Secure Coding 🔐 Conference Speaker – dozens of recorded talks on YouTube Founder of European SoMware CraMers (5500+ members) 🔥 Free monthly webinars, 1 hour a@er work 👉 victorrentea.ro/community Past events on youtube.com/vrentea Father of 👧👦, servant of a 🐈, weekend gardener 🌼 h"ps://VictorRentea.ro
3. 263 From the Agile (2001) Ideology ... Emergent Design While we keep shipping shit, the design of the system design will naturally improve (as opposed to large up-front design that caused overengineering in waterfall) working so8ware
4. 264 Wri$ng Unit Tests gives you those triggers! Emergent Design that never emerged We need triggers to start improving the design! I f t e s t s a r e h a r d t o w r i t e , t h e d e s i g n s u c k s
5. 265 Kent Beck Creator of Extreme Programming (XP) in 1999 the most technical style of Agile Inventor of TDD Author of JUnit Father of Unit Tes3ng
6. 266 1. Passes all Tests 💪 2. Expresses Intent = SRP, Domain Names 2. No Duplica;on of Logic = DRY🌵 3. Keep it Simple = KISS💋 Rules of Simple Design by Kent Beck https://martinfowler.com/bliki/BeckDesignRules.html è Design Feedback 💎
7. 270 https://martinfowler.com/articles/practical-test-pyramid.html E2E Tes1ng Pyramid Test a thin slice of behavior MOCKS Test a group of modules Test everything as whole ⭐ Deep Edge Case ⭐ Cri5cal Business Flow (eg. checkout) overlapping is expected surface is propor?onal to quan?ty
8. 271 MOCKS 💖 / 🤬
9. 272 Why we 💖 Mocks Isolated Tests from external systems Fast 👑 no framework, DB, external API Test Less Logic when tes7ng high complexity 😵💫 è Alterna7ves: - In-mem DB - Testcontainers 🐳 - WireMock, ..
10. 273 public computePrices(!!...) { !// A for (Product product : products) { +1 !// B if (price !== null) { +1 !// C } for (Coupon coupon : customer.getCoupons()) { +1 if (coupon.autoApply() +1 !&& coupon.isApplicableFor(product, price) +1 !&& !usedCoupons.contains(coupon)) { +1 !// D } } } return !!...; } Code Complexity - Cycloma(c - Cogni(ve (Sonar)
11. 275 MOCK
12. 276 Why we 💖 Mocks Isolated Tests from external systems Fast 👑 no framework, DB, external API Test Less Logic when tes7ng high complexity 😵💫 è Alterna7ves: - In-mem DB - Testcontainers 🐳 - WireMock, .. 🤨 Chea&ng James Coplien in hEps://rbcs-us.com/documents/Why-Most-Unit-Tes7ng-is-Waste.pdf
13. 277 Why we 🤬 Mocks Uncaught Bugs 😱 despite 1000s of ✅GREEN tests (lock, tap, doors, umbrella) Fragile Tests 💔 that break at any refactoring Unreadable Tests 😵💫 eg. a test using ≥ 5 mocks è
14. 280 WHAT AM I TESTING HERE ? syndrome
15. 281 Test has 20 lines full of mocks 😵💫 😡 BURN THE TEST! bad cost/benefit ra5o HONEYCOMB TESTS Integra5on test this! WHAT AM I TESTING HERE ? syndrome Tested prod code has 4 lines 😩 f(..) { a = api.fetchB(repo1.find(..).getBId()); d = service.createD(a,b,repo2.find(..)); repo3.save(d); mq.send(d.id); } g(dto) { repo2.save(mapper.fromDto(dto, repo1.find(..))); } SIMPLIFY PRODUCTION "Remove Middle-Man" (inline method)
16. 282 https://martinfowler.com/articles/practical-test-pyramid.html End-to-end The Tes7ng Pyramid For Monoliths
17. 283 §The Legacy Monolith has - Terrible complexity behind only a few entry points è Only Unit Tests can cover deep corners of logic è §Microservices expose more APIs (test surfaces) ... hold less complexity, but more configura7on è Test more at API-level (aka Integra7on) = Honeycomb Tes$ng Strategy
18. 284 Integra3on (one microservice) Integrated (en-re ecosystem) Start up all microservices (in dockers / staging) Expensive, slow, flaky tests Required for business-cri5cal flows (eg. checkout) Cover one microservice (eg @SpringBootTest) Use for: Every flow of the system ⭐⭐⭐ Keep tests isolated without mocks Cover 1..few classes (Social Unit Testsè) Use for: naturally isolated pieces with high complexity. @Mock are allowed 🤨 Honeycomb Tes,ng Strategy Testcontainers 🐳 WireMock &co Contract Tests (Pact, SCC) DB ES Kafka ... API many tests on one entry point h,ps://engineering.atspo8fy.com/2018/01/tes8ng-of-microservices/ Test DSL (helper code) Implementa$on Detail (a role) complexity decouple and test in isola<on
19. 286 Test manageable complexity without mocks #0 Integra,on Test one flow end-to-end The hard part: When you change a 7ny part, understand and test the whole flow 😱 eg: don't test alone simple mappers (toDto)
20. 287 Unit Tests give you most Design Feedback 💎 More Complexity => BeBer Design Implementa&on Detail Tests 👍 MOCKS complexity
21. 288 The Dawn of Mocks (2004)
22. 289 The Dawn of Mocks (2004) h7p://jmock.org/oopsla2004.pdf
23. 290 BAD HABIT Mock Roles, not Objects h=p://jmock.org/oopsla2004.pdf You implement a new feature > ((click in UI/postman)) > It works > 🎉 Oups!! I forgot about unit-tests 😱 ...then you write unit tests mocking all the dependencies of the prod code you wrote ↓ Few years later: "My tests are fragile and impede refactoring!" Contract-Driven Design Before mocking a dependency, clarify its responsibility = AZer you mock an API ❄ changing it = pain 😭
24. 291 write Social Unit Tests for "components" (groups of objects) with clear responsibiliJes A B ✅ Internal Refactoring won't break such tests Instead of fine-grained Solitary Unit Tests tesJng each class in isolaJon,
25. 292 "Unit Tes5ng means mocking all dependencies of a class" - common belief WRONG! "It's perfectly fine for unit tests to talk to databases and filesystems!"- Ian Cooper in TDD, Where Did It All Go Wrong Unit Tes7ng = ? Robust Unit TesJng requires idenJfying responsibiliJes
26. 294 Unit Tes<ng tells you to simplify methods and data structures #1
27. 295 var bigObj = new BigObj(); bigObj.setA(a); bugObj.setB(b); prod.method(bigObj); Tests must create bigObj just to pass two inputs🫤 method(bigObj) MUTABLE DATA 😱 in 2023? using only 2 of the 15 fields in bigObj method(a, b) Precise Signatures prod.method(a, b); Also, simpler tests: Pass only necessary data to func7ons ✅ when(bigObj.getA()).thenReturn(a); ⛔ Don't Mock GeEers ⛔ ✅ Mock behavior, not data prod.horror(new ABC(a, b, c)); horror(abc) Parameter Object class ABC {a,b,c} When tes7ng highly complex logic, introduce a ⛔ Don't return Mocks from Mocks⛔ when(bigObj.getA()).thenReturn(mockA);
28. 296 Constrained Objects = data structures that throw excep7ons on invalid data (eg required fields) Ø Mutable (eg Domain En77es, Aggregates) Ø Immutable❤ (Value Objects) 🏰
29. 297 ❶ A Constrained Object gets Large ↓ Object Mother PaCern TestData.aCustomer(): Customer (valid) coupling Break Domain En77es InvoicingCustomer | ShippingCustomer in separate Bounded Contexts packages > modules > microservices ❷ Integra$on and Social Unit Tests❤ require heavy data inputs * h=ps://mar<nfowler.com/bliki/ObjectMother.html (2006) Same Object Mother used in different ver+cals invoicing | shipping ↓ Split Object Mother per ver$cal InvoicingTestData | ShippingTestData Crea7ng valid test data gets cumbersome CREEPY A large class shared by many tests Don't change it! Just add methods, or tweak results ✅ TestData.charles(): Customer (a persona) 🏰
30. 299 Tests require detailed understanding of: External APIs: to populate/assert DTOs The Library: to mock it ... externalApi.call(apiDetails); ... externalDto.getFooField() ... Lib.use(mysteriousParam,...) Tests are expressed in your Domain #respect your tests Agnos7c Domain Isolate complex logic from external world via Adapters ... clientAdapter.call(domainStuff) ... domainObject.getMyField() ... libWrapper.use(😊) Your complex logic directly uses External APIs or heavy libraries:
31. 300 applica3on / infra My DTOs External API External DTOs Client External Systems ApplicaHon Service Simplified Onion Architecture (more in my "Clean Pragma<c Architecture" at Devoxx Ukraine 2021 on YouTube) Value Object EnHty id Domain Service Domain Service agnos5c domain Repo IAdapter Adapter Dependency Inversion ⛔ ⛔ IWrapper W rapper Dependency Inversion Ugly Invasive Library Domain complexity kept inside
32. 301 #1 Unit Tes<ng encourages Precise Signatures Tailored Data Structures Agnos<c Domain
33. 302 #2 Unit Tes<ng puts design pressure on Highly Complex Logic 🧠
34. 303 class Big { f() { //complex g(); } g() { //complex } } Inside the same class, a complex func7on f() calls a complex g() g() is complex => unit-tested separately When tes7ng f(), can I avoid entering g()? Can I mock a local method call? class HighLevel { LowLevel low; f() {//complex low.g(); } } class LowLevel { g() {/*complex*/} } ↓ Par(al Mock (@Spy) Hard to maintain tests: Which method is real, which is mocked?🤯 Split by Layers of Abstrac7on (high-level policy vs low-level details) Only when tes?ng Legacy Code If splijng the class doesn't feel right, test f() + g() together with social unit tests ⛔
35. 304 class HighLevel { LowLevel low; f() {//complex low.g(); } } Split by Layers of Abstrac7on = ver&cal split of a class class LowLevel { g() {/*complex*/} } horizontal è
36. 305 class Wide { A a; B b; //+more dependencies f() {..a.a()..} g() {..b.b()..} } @ExtendWith(MockitoExtension) class WideTest { @Mock A a; @Mock B b; @InjectMocks Wide wide; // 5 tests for f() // 7 tests for g() } ↓ Split test class in FTest, GTest (rule: the @Before should be used by all tests) Unrelated complex methods in the same class use different sets of dependencies: class ComplexF { A a; f() { ..a.a().. } } class ComplexG { B b; g() { ..b.b().. } } @BeforeEach void sharedFixture() { when(a.a()).then... when(b.b()).then... } Split Unrelated Complexity Later: what part of the before is used by my failed test? ** Mockito (since v2.0) throws UnnecessaryStubbingException if a when..then is not used by a @Test, when using MockitoExtension = Unmaintainable Tests 🤔 FIXTURE CREEP test setup DRY https://www.davidvlijmincx.com/posts/setting_the_strictness_for_mockito_mocks/ not used by these tests
37. 306 Split Unrelated Complexity class ComplexF { A a; f() { ..a.a().. } } class ComplexG { B b; g() { ..b.b().. } } horizontally ↔
38. 309 ver&cally ↕ Tests help us to break complexity horizontally ↔ #2 clear roles by
39. 310 #3 Unit Tes<ng promotes Func,onal Programming pure func&ons & immutable objects
40. 313 1) Has no Side-Effects (doesn't change anything) INSERT, POST, send message, field changes, files 2) Returns Same Output for Same Inputs (no external data source) GET, SELECT, Jme, random, … Pure Func)on aka "Referen*al Transparency" Just compute a value 𝑒𝑔: 𝑀𝑎𝑡ℎ𝑒𝑚𝑎𝑡𝑖𝑐𝑎𝑙 𝐹𝑢𝑛𝑐𝑡𝑖𝑜𝑛𝑠: 𝑓(𝑥,𝑦) = 𝑥2 + 𝑦
41. 314 No Network or files No Changes to Data No ,me/random Pure Func)ons immutable objects❤ (simplified defini5on)
42. 316 a = repo1.findById(..) b = repo2.findById(..) c = api.call(..) 🤯complexity🤯 repo3.save(d); mq.send(d.id); Complex logic using many dependencies (eg: computePrice, applyDiscounts) Many tests using lots of mocks when(..).thenReturn(a); when(..).thenReturn(b); when(..).thenReturn(c); prod.complexAndCoupled(); verify(..).save(captor); d = captor.get(); assertThat(d)... verify(..).send(...); x 15 tests =😖 ✅ Simpler tests (less mocks) d = prod.pure(a,b,c); assertThat(d)... Reduce Coupling of Complex Logic ✅ Simpler produc7on code D pure(a,b,c) { 🤯complexity🤯 return d; } 5 mocks
43. 317 © VictorRentea.ro a training by Complexity State Muta3on DB Impera7ve Shell API call Files Dependencies Complex Logic Func7onal Core
44. 318 © VictorRentea.ro a training by Complexity State Muta3on DB API call Files Dependencies Complex Logic / Segrega7on Impera7ve Shell Func7onal Core
45. 319 © VictorRentea.ro a training by Func&onal Core Impera&ve Shell Extract heaviest complexity as pure funcJons / Segrega7on Impera7ve Shell Func7onal Core
46. 320 method(Mutable order, discounts) { ds.applyDiscounts(order, discounts); var price = cs.computePrice(order); return price; } ... but you use mutable objects Swapping two lines causes bugs in produc7on ❌ despite 4000 ✅ tests You have 4.000 unit tests, 100% test coverage 😲 👏 ↓ Paranoid Tes7ng (InOrder can verify method call order) Immutable Objects method(Immutable order, d) { var discountedOrder = ds.applyDiscounts(order, d); var price = cs.computePrice(discountedOrder); return price; } TEMPORAL COUPLING Swapping the two lines ❌ breaks compila$on
47. 322 1. Collapse Middle-Man vs "What am I tes,ng here?" Syndrome or test more: 2. Honeycomb Tes7ng Strategy: more Integra,on Tests over Fragile Unit Tests 3. Precise Signatures: simpler arguments, be?er names 4. Dedicated Data Structures vs Creepy Object Mother 5. Agnos7c Domain vs using external APIs or Libraries in complex logic 6. Split Complexity by Layers of Abstrac7on ↕ vs Par,al Mocks (@Spy) 7. Split Unrelated Complexity ↔ vs Fixture Creep (bloated @Before) 8. Clarify Roles, Social Unit Tests vs blindly @Mock all dependencies 9. Decouple Complexity in Pure Func7ons vs Many tests full of mocks 10.Immutable Objects vs Temporal Coupling Design Hints from Tests
48. 323 Testable Design is Good Design
49. 324 <meframe for developing your feature When do you start wri0ng tests? ✅ understand the problem => early ques3ons to biz ✅ early design feedback 💎 💎 💎 ✅ test coverage => courage to refactor later BDD (.feature) too late TDD Good enough TDD!
50. Wri0ng unit tests early increases fric0on with careless design
51. 326 Unit Tes0ng Reading Guide (for later) 1] Classic TDD⭐⭐⭐ (mock-less) https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530 Mock Roles, not Objects ⭐⭐⭐: http://jmock.org/oopsla2004.pdf "Is TDD Dead?" https://martinfowler.com/articles/is-tdd-dead/ Why Most Unit Testing is Waste (James Coplien): https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf vs Integrated Tests are a Scam(J Brains): https://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam 2] London TDD⭐⭐⭐ (mockist) https://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627 3] Patterns⭐ https://www.amazon.com/Art-Unit-Testing-examples/dp/1617290890 4] https://www.amazon.com/xUnit-Test-Patterns-Refactoring-Code/dp/0131495054/ 5] (skip through) https://www.amazon.com/Unit-Testing-Principles-Practices-Patterns
52. Your tests are trying to tell you something ... 10 design hints you were missing Read the article: https://victorrentea.ro/blog/design-insights-from-unit-testing/ Ways Victor can help you:: https://victorrentea.ro Stay connected, and join us: