Your Unit Tests are Trying to tell you something @ DevoxxUK23

If tests are hard to write, the production design is crappy – goes an old saying. Indeed, writing unit tests gives you one of the most comprehensive, yet brutal, feedback about the design of your production code, but if it comes too late, many developers can’t stand it anymore and they will either stop testing or test more superficially. At the other extreme, others struggle to write contrived, fragile tests full of mocks that end up frustrating more than helping them. This talk reviews the main hints that unit tests provide you, from the most obvious improvements to some of the most subtle design principles.

Transcript

1. Your tests are trying to tell you something ... 10 design hints you were missing ↓ Read the article: https://victorrentea.ro/blog/design-insights-from-unit-testing/ About the speaker: https://victorrentea.ro
2. 👉 victorrentea.ro/training-offer Hi, I'm Victor Rentea 🇷🇴 PhD(CS) Java Champion, 17 years of code, code, code, code, code.... Consultant & Trainer: 5000 developers of 100+ companies in EU: ❤ Clean Code, Architecture, Unit Tes3ng 🛠 Frameworks: Spring, Hibernate, WebFlux ⚡ Java Performance, Secure Coding 🔐 Conference Speaker – dozens of recorded talks on YouTube Founder of European SoMware CraMers (5500+ members) 🔥 Free monthly webinars, 1 hour a@er work 👉 victorrentea.ro/community Past events on youtube.com/vrentea Father of 👧👦, servant of a 🐈, weekend gardener 🌼 h"ps://VictorRentea.ro
3. 3 VictorRentea.ro From the Agile Ideology ... Emergent Design while we keep shipping shit working so/ware fast, the design of the system will naturally evolve by itself (as opposed to large up-front design that caused overengineering in waterfall)
4. 4 VictorRentea.ro Wri$ng Tests tells you when to improve the design Emergent Design that never emerged 😩 We need triggers! When should I refine the design?
5. 5 VictorRentea.ro Kent Beck Creator of Extreme Programming (XP) in 1999 the most technical style of Agile Inventor of TDD Author of JUnit Father of Unit Tes3ng
6. 6 VictorRentea.ro 1. Passes all Tests 💪 2. Expresses Intent = SRP, Domain Names 2. No Duplica;on of Logic = DRY🌵 3. Keep it Simple = KISS💋 Rules of Simple Design by Kent Beck https://martinfowler.com/bliki/BeckDesignRules.html è Design Feedback 💎
7. 7 VictorRentea.ro
8. 8 VictorRentea.ro History of Tes,ng: Ice-Cream Cone What's this?
9. 9 VictorRentea.ro Testing Anti-Pattern 30 manual testers 🧔🧔🧔🧔 🧔🧔🧔 🧔🧔🧔🧔🧔 ... 20% end-to-end coverage using GUI robots, Selenium,.. absent (old frameworks) scarce unit-tests, 10% coverage Ice-Cream Cone History of Tes,ng: Reality in many Successful Monoliths today:
10. 10 VictorRentea.ro https://martinfowler.com/articles/practical-test-pyramid.html E2E Tes1ng Pyramid Test a thin slice of behavior MOCKS Test a group of modules Test everything as whole ⭐ Deep Edge Case ⭐ Cri,cal Business Flow (eg. checkout) overlapping is expected surface is propor9onal to quan9ty
11. 11 VictorRentea.ro MOCKS 💖 / 🤬
12. 12 VictorRentea.ro Why we 💖 Mocks Isolated Tests from external systems Fast 👑 no framework, DB, external API Test Less Logic when tes$ng high complexity 😵💫 è Alterna$ves: - In-mem DB - Testcontainers 🐳 - WireMock, .. 🤨 Chea%ng↓ * James Coplien in h/ps://rbcs-us.com/documents/Why-Most-Unit-Tes?ng-is-Waste.pdf
13. 13 VictorRentea.ro public computePrices(!!...) { !// A for (Product product : products) { +1 !// B if (price !== null) { +1 !// C } for (Coupon coupon : customer.getCoupons()) { +1 if (coupon.autoApply() +1 !&& coupon.isApplicableFor(product, price) +1 !&& !usedCoupons.contains(coupon)) { +1 !// D } } } return !!...; } Code Complexity - Cycloma(c - Cogni(ve (Sonar)
14. 14 VictorRentea.ro Logic under test Tes5ng Complex Code Many execu$on paths through code => lots of tests Test Test Test Test Test Test Test CC=5 CC=6 f(a) g(b) calls f() calling g() together have a CC of max ... Too Many (to cover all branches) Too Heavy (setup and input data) Integra%on Tests for f+g are... 30 Test Test Test Test Test Test Test Test Test Test Test Test
15. 15 VictorRentea.ro MOCK
16. 16 VictorRentea.ro Why we 🤬 Mocks Uncaught Bugs 😱 despite 1000s of GREEN✅ tests: lock, tap, doors, umbrella 🤣 Fragile Tests 💔 that break at any refactoring Unreadable Tests 😵💫 eg. a test using 5+ mocks è
17. 17 VictorRentea.ro WHAT AM I TESTING HERE ? syndrome
18. 18 VictorRentea.ro Test has 20 lines full of mocks 😵💫 😡 BURN THE TEST! bad cost/benefit ra,o HONEYCOMB TESTS Integra,on test this! WHAT AM I TESTING HERE ? syndrome Tested prod code has 4 lines 😩 f(..) { a = api.fetchB(repo1.find(..).getBId()); d = service.createD(a,b,repo2.find(..)); repo3.save(d); mq.send(d.id); } g(dto) { repo2.save(mapper.fromDto(dto, repo1.find(..))); } SIMPLIFY PRODUCTION Collapse Middle-Man (useless method)
19. 19 VictorRentea.ro https://martinfowler.com/articles/practical-test-pyramid.html End-to-end The Tes7ng Pyramid For Monoliths
20. 20 VictorRentea.ro §The Legacy Monolith ("Big ball of mud") has - Terrible complexity behind few entry points è Many Unit Tests are needed for the deeper corners of logic è §Tes(ng Microservices is different - Huge complexity in a single microservice = bad practice è Break it ! è More APIs, hiding a decent amount of complexity (🙏) è Possible to test more at the API level = Honeycomb Tes0ng Strategy
21. 21 VictorRentea.ro Integra4on test one microservice endpoint end-to-end Integrated test the en0re ecosystem Implementa4on Detail Start up all microservices (in dockers / staging) Expensive, slow, flaky tests Required for business-criJcal flows (eg. checkout) Start up one microservice Use for: Every flow of the system Keep tests isolated without mocks Instan0ate several classes. Use for: naturally isolated parts of code with high complexity. Mocks are tolerated (but sJll avoided): Test components with clear roles using social unit tests Honeycomb Tes7ng Strategy Testcontainers 🐳 WireMock Contract Tests (Pact, SCC) DB ES Kafka ... API many tests on one entry point h,ps://engineering.atspo8fy.com/2018/01/tes8ng-of-microservices/ complexity decouple and test in isola/on
22. 22 VictorRentea.ro Test manageable complexity without mocks #0 Integra.on test one flow end-to-end
23. 23 VictorRentea.ro Give you most Design Feedback 💎 More Complexity => BeDer Design Implementa%on Detail Tests 👍 MOCKS
24. 24 VictorRentea.ro The dawn of Mocks (2004)
25. 25 VictorRentea.ro Mock Roles, not Objects h"p://jmock.org/oopsla2004.pdf The dawn of Mocks (2004)
26. 26 VictorRentea.ro The dawn of Mocks (2004)
27. 27 VictorRentea.ro BAD HABIT Mock Roles, not Objects h1p://jmock.org/oopsla2004.pdf You implement a new feature > ((click in UI/postman)) > It works > 🎉 Oups!! I forgot about unit-tests 😱 ...then you write unit tests mocking all the dependencies of the prod code you wrote ↓ Few years later: "My tests are fragile and impede refactoring!" Contract-Driven Design Before mocking a dependency, clarify its responsibility = Changing an API you mocked is painful 😭
28. 28 VictorRentea.ro write Social Unit Tests for "components" (groups of objects) with clear responsibili8es A B ✅ Internal refactoring won't break these tests Instead of solitary fine-grained unit tests mocking every dependency,
29. 29 VictorRentea.ro "Unit Tes=ng means mocking all dependencies of a class" - common belief WRONG! "It's perfectly fine for unit tests to talk to databases and filesystems!"- Ian Cooper in his Talk Unit Tes$ng = ? Robust Unit Tes$ng requires iden$fying responsibili$es
30. 30 VictorRentea.ro Unit Tes>ng tells you to simplify methods and data structures #1
31. 31 VictorRentea.ro var bigObj = new BigObj(); bigObj.setA(a); bugObj.setB(b); prod.method(bigObj); Tests must create bigObj just to pass two inputs🫤 method(bigObj) MUTABLE DATA 😱 in 2023? using only 2 of the 15 fields in bigObj method(a, b) Precise Signatures prod.method(a, b); Also, simpler tests: Pass only necessary data to func$ons ✅ when(bigObj.getA()).thenReturn(a); ⛔ Don't Mock Ge_ers ⛔ ✅ Mock behavior, not data prod.method(new ABC(a, b, c)); method(abc) Parameter Object When tes$ng highly complex logic, introduce a ⛔ Don't return Mocks from Mocks⛔ when(bigObj.getA()).thenReturn(mockA); OMG! They sBck!
32. 32 VictorRentea.ro 🏰 Constrained Objects = data structures that guard their internal consistency by throwing excep$ons, (eg required fields) Ø Mutable (eg Domain En$$es, Aggregates) Ø Immutable❤ (Value Objects)
33. 33 VictorRentea.ro ❷ A group of classes has a clear role and manageable complexity ❶ A Constrained Object🏰 gets Large Object Mother Pa?ern TestData.aCustomer(): Customer (valid) coupling Break Domain En77es in separate Bounded Contexts packages > modules > microservices invoicing | shipping è Tes$ng the en$re group with a social unit-test✅ requires heavier data inputs * h1ps://mar/nfowler.com/bliki/ObjectMother.html (2006) Same Object Mother used in different ver+cals invoicing | shipping è Split Object Mother per ver4cal InvoicingTestData | ShippingTestData Crea$ng valid test data gets cumbersome CREEPY A large class shared by many tests Don't change it. Add methods, or tweak results ✅ TestData.charles(): Customer (a persona)
34. 34 VictorRentea.ro Your complex logic directly uses External APIs or heavy libraries: Unit-tes$ng your logic requires understanding the seman$cs of: The External API: to populate/assert DTOs The Library: to mock it ... externalApi.call(apiDetails); ... dto.getStrangeField() ... Lib.use(mysteriousParam,...) Unit Tests should speak your Domain Model (mock the Adapter when tes/ng complex domain logic) Unit Tests are first-class ci$zens of your project #respect your tests Agnos7c Domain Isolate complex logic from the outside world ... clientAdapter.call(domainStuff) ... domainObject.getMyField() ... libAdapter.use(😊)
35. 35 VictorRentea.ro applica3on / infra Value Object En0ty id Domain Service Domain Service agnos/c domain My DTOs External API External DTOs Client External Systems Applica0on Service Controller Repo Clean, Pragma/c Architecture at Devoxx Ukraine 2021 Curious for more? ↓ IAdapter Adapter Dependency Inversion ⛔ ⛔ Simplified Onion Architecture Interf W rapper f Dependency Inversion Ugly Invasive Library Domain complexity is kept inside for easier tes3ng
36. 36 VictorRentea.ro #1 Unit Tes>ng encourages Precise Signatures Tailored Data Structures Agnos>c Domain
37. 37 VictorRentea.ro #2 Unit Tes>ng puts design pressure on Highly Complex Logic 🧠
38. 38 VictorRentea.ro class Big { f() { //complex g(); } g() { //complex } } Inside the same class, a complex func$on f() calls a complex g() g() is complex => unit-tested separately When tes$ng f(), can I avoid entering g()? Can I mock a local method call? class HighLevel { LowLevel low; f() {//complex low.g(); } } class LowLevel { g() {/*complex*/} } ↓ Par(al Mock (@Spy) Hard to maintain tests: Which method is real, which is mocked?🤯 Split by Layers of Abstrac7on (high-level policy vs low-level details) Tolerable tes+ng Legacy Code If splijng the class doesn't feel right, test f() + g() together with bigger tests
39. 39 VictorRentea.ro class HighLevel { LowLevel low; f() {//complex low.g(); } } Split by Layers of Abstrac7on = ver%cal split of a class class LowLevel { g() {/*complex*/} }
40. 40 VictorRentea.ro class Wide { A a; B b; //+more dependencies f() {..a.a()..} g() {..b.b()..} } @ExtendWith(MockitoExtension) class WideTest { @Mock A a; @Mock B b; @InjectMocks Wide wide; // 5 tests for f() // 4 tests for g() } ↓ Split test class in ComplexFTest, ..G.. (rule: the before should be used by all tests) Complex methods in the same class use different sets of dependencies: class ComplexF { A a; f() { ..a.a().. } } class ComplexG { B b; g() { ..b.b().. } } @BeforeEach void fixture() { when(a.a()).then... when(b.b()).then... } Split Unrelated Complexity Later: what part of the before is used by my failed test? ** Mockito (since v2.0) throws UnnecessaryStubbingException if a when..then is not used by a @Test, when using MockitoExtension = Unmaintainable Tests 🤔 FIXTURE CREEP test setup DRY https://www.davidvlijmincx.com/posts/setting_the_strictness_for_mockito_mocks/ not used by these tests
41. 41 VictorRentea.ro Split Unrelated Complexity class ComplexF { A a; f() { ..a.a().. } } class ComplexG { B b; g() { ..b.b().. } } horizontally ↔
42. 42 VictorRentea.ro ver%cally ↕ Tests help us to break complexity horizontally ↔ When should we follow those hints? clear roles
43. 43 VictorRentea.ro The dawn of Mocks (2004):
44. 44 VictorRentea.ro ver%cally ↕ Tests help us to break complexity horizontally ↔ clear roles #2 by
45. 45 VictorRentea.ro #3 Unit Tes>ng promotes Func.onal Programming pure func%ons & immutable objects
46. 46 VictorRentea.ro 1) Has no Side-Effects (doesn't change anything) INSERT, POST, send message, field changes, files 2) Returns Same Output for Same Inputs (no external source of data) GET, SELECT, current 8me, random, … Pure Func)on aka "Referen?al Transparency" Just compute a value 𝑒𝑔: 𝑀𝑎𝑡ℎ𝑒𝑚𝑎𝑡𝑖𝑐𝑎𝑙 𝐹𝑢𝑛𝑐𝑡𝑖𝑜𝑛𝑠: 𝑓(𝑥,𝑦)=𝑥^2+𝑦
47. 47 VictorRentea.ro No Network or files No Changes to Data No .me/random Pure Func)ons immutable objects❤ (a simplified definiJon)
48. 49 VictorRentea.ro a = repo1.findById(..) b = repo2.findById(..) c = api.call(..) 🤯complexity🤯 repo3.save(d); mq.send(d.id); Complex logic using many dependencies (eg: computePrice, applyDiscounts) Many tests using lots of mocks when(..).thenReturn(a); when(..).thenReturn(b); when(..).thenReturn(c); prod.complexAndCoupled(); verify(..).save(captor); d = captor.get(); assertThat(d)... verify(..).send(...); x15=😖 ✅ Easier to test, with less mocks d = prod.pure(a,b,c); assertThat(d)... Reduce Coupling of Complex Logic ✅ Easier to understand D pure(a,b,c) { 🤯complexity🤯 return d; }
49. 50 © VictorRentea.ro a training by Complexity Func%onal Core Impera%ve Shell / Func%onal Core Segrega%on State Muta3on DB Impera%ve Shell API call Files Dependencies Complex Logic
50. 51 © VictorRentea.ro a training by Func%onal Core Impera%ve Shell Impera%ve Shell / Func%onal Core Segrega%on Extract heaviest complexity as pure func5ons
51. 52 VictorRentea.ro method(Mutable order, discounts) { ds.applyDiscounts(order, discounts); var price = cs.computePrice(order); return price; } ... but you use mutable objects Swapping two lines causes bugs in produc@on ❌ despite 4000 ✅ tests You have 4.000 unit tests, 100% test coverage 😲 👏 ↓ Paranoid Tes@ng (verify method call order) Immutable Objects method(Immutable order, d) { var discountedOrder = ds.applyDiscounts(order, d); var price = cs.computePrice(discountedOrder); return price; } TEMPORAL COUPLING Swapping the two lines ❌ breaks compila4on
52. 54 VictorRentea.ro 1. Collapse Middle-Man vs "What am I tes,ng here?" Syndrome 2. Honeycomb Tes7ng Strategy: more Integra,on Tests than Fragile Unit Tests 3. Precise Signatures: less arguments 4. Dedicated Data Structures vs Creepy Object Mother 5. Agnos7c Domain vs using APIs or Libraries in complex logic 6. Split Complexity by Layers of Abstrac7on ↕ vs Par,al Mocks (@Spy) 7. Split Unrelated Complexity ↔ vs Fixture Creep (bloated setup) 8. Clarify Roles, Social Unit Tests vs blindly @Mock all dependencies 9. Decouple Complexity in Pure Func7ons vs Tests full of mocks 10.Immutable Objects vs Temporal Coupling Design Hints from Tests
53. 55 VictorRentea.ro Testable Design is Good Design
54. 56 VictorRentea.ro >meframe for developing your feature When do you start wri%ng tests? ✅ understand the problem => early ques3ons to biz ✅ early design feedback 💎 💎 💎 ✅ real test coverage => courage to refactor later BDD (.feature) too late TDD Good enough TDD!
55. Wri%ng unit tests early increases fric%on with careless design
56. 58 VictorRentea.ro Unit Tes%ng Reading Guide (for later) 1] Classic TDD⭐⭐⭐ (mock-less) https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530 Mock Roles, not Objects ⭐⭐⭐: http://jmock.org/oopsla2004.pdf "Is TDD Dead?" https://martinfowler.com/articles/is-tdd-dead/ Why Most Unit Testing is Waste (James Coplien): https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf vs Integrated Tests are a Scam(J Brains): https://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam 2] London TDD⭐⭐⭐ (mockist) https://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627 3] Patterns⭐ https://www.amazon.com/Art-Unit-Testing-examples/dp/1617290890 4] https://www.amazon.com/xUnit-Test-Patterns-Refactoring-Code/dp/0131495054/ 5] (skip through) https://www.amazon.com/Unit-Testing-Principles-Practices-Patterns
57. Your tests are trying to tell you something ... 10 design hints you were missing Read the article: https://victorrentea.ro/blog/design-insights-from-unit-testing/ About the speaker: https://victorrentea.ro at è Stay connected. Join: