Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

Your Unit Tests are Trying to tell you somethin...

Your Unit Tests are Trying to tell you something @ DevoxxUK23

If tests are hard to write, the production design is crappy – goes an old saying. Indeed, writing unit tests gives you one of the most comprehensive, yet brutal, feedback about the design of your production code, but if it comes too late, many developers can’t stand it anymore and they will either stop testing or test more superficially. At the other extreme, others struggle to write contrived, fragile tests full of mocks that end up frustrating more than helping them. This talk reviews the main hints that unit tests provide you, from the most obvious improvements to some of the most subtle design principles.

Victor Rentea

May 10, 2023
Tweet

More Decks by Victor Rentea

Other Decks in Technology

Transcript

  1. Your tests are trying to tell you something ... 10

    design hints you were missing ↓ Read the article: https://victorrentea.ro/blog/design-insights-from-unit-testing/ About the speaker: https://victorrentea.ro
  2. 👉 victorrentea.ro/training-offer Hi, I'm Victor Rentea 🇷🇴 PhD(CS) Java Champion,

    17 years of code, code, code, code, code.... Consultant & Trainer: 5000 developers of 100+ companies in EU: ❤ Clean Code, Architecture, Unit Tes3ng 🛠 Frameworks: Spring, Hibernate, WebFlux ⚡ Java Performance, Secure Coding 🔐 Conference Speaker – dozens of recorded talks on YouTube Founder of European SoMware CraMers (5500+ members) 🔥 Free monthly webinars, 1 hour a@er work 👉 victorrentea.ro/community Past events on youtube.com/vrentea Father of 👧👦, servant of a 🐈, weekend gardener 🌼 h"ps://VictorRentea.ro
  3. 3 VictorRentea.ro From the Agile Ideology ... Emergent Design while

    we keep shipping shit working so/ware fast, the design of the system will naturally evolve by itself (as opposed to large up-front design that caused overengineering in waterfall)
  4. 4 VictorRentea.ro Wri$ng Tests tells you when to improve the

    design Emergent Design that never emerged 😩 We need triggers! When should I refine the design?
  5. 5 VictorRentea.ro Kent Beck Creator of Extreme Programming (XP) in

    1999 the most technical style of Agile Inventor of TDD Author of JUnit Father of Unit Tes3ng
  6. 6 VictorRentea.ro 1. Passes all Tests 💪 2. Expresses Intent

    = SRP, Domain Names 2. No Duplica;on of Logic = DRY🌵 3. Keep it Simple = KISS💋 Rules of Simple Design by Kent Beck https://martinfowler.com/bliki/BeckDesignRules.html è Design Feedback 💎
  7. 9 VictorRentea.ro Testing Anti-Pattern 30 manual testers 🧔🧔🧔🧔 🧔🧔🧔 🧔🧔🧔🧔🧔

    ... 20% end-to-end coverage using GUI robots, Selenium,.. absent (old frameworks) scarce unit-tests, 10% coverage Ice-Cream Cone History of Tes,ng: Reality in many Successful Monoliths today:
  8. 10 VictorRentea.ro https://martinfowler.com/articles/practical-test-pyramid.html E2E Tes1ng Pyramid Test a thin slice

    of behavior MOCKS Test a group of modules Test everything as whole ⭐ Deep Edge Case ⭐ Cri,cal Business Flow (eg. checkout) overlapping is expected surface is propor9onal to quan9ty
  9. 12 VictorRentea.ro Why we 💖 Mocks Isolated Tests from external

    systems Fast 👑 no framework, DB, external API Test Less Logic when tes$ng high complexity 😵💫 è Alterna$ves: - In-mem DB - Testcontainers 🐳 - WireMock, .. 🤨 Chea%ng↓ * James Coplien in h/ps://rbcs-us.com/documents/Why-Most-Unit-Tes?ng-is-Waste.pdf
  10. 13 VictorRentea.ro public computePrices(!!...) { !// A for (Product product

    : products) { +1 !// B if (price !== null) { +1 !// C } for (Coupon coupon : customer.getCoupons()) { +1 if (coupon.autoApply() +1 !&& coupon.isApplicableFor(product, price) +1 !&& !usedCoupons.contains(coupon)) { +1 !// D } } } return !!...; } Code Complexity - Cycloma(c - Cogni(ve (Sonar)
  11. 14 VictorRentea.ro Logic under test Tes5ng Complex Code Many execu$on

    paths through code => lots of tests Test Test Test Test Test Test Test CC=5 CC=6 f(a) g(b) calls f() calling g() together have a CC of max ... Too Many (to cover all branches) Too Heavy (setup and input data) Integra%on Tests for f+g are... 30 Test Test Test Test Test Test Test Test Test Test Test Test
  12. 16 VictorRentea.ro Why we 🤬 Mocks Uncaught Bugs 😱 despite

    1000s of GREEN✅ tests: lock, tap, doors, umbrella 🤣 Fragile Tests 💔 that break at any refactoring Unreadable Tests 😵💫 eg. a test using 5+ mocks è
  13. 18 VictorRentea.ro Test has 20 lines full of mocks 😵💫

    😡 BURN THE TEST! bad cost/benefit ra,o HONEYCOMB TESTS Integra,on test this! WHAT AM I TESTING HERE ? syndrome Tested prod code has 4 lines 😩 f(..) { a = api.fetchB(repo1.find(..).getBId()); d = service.createD(a,b,repo2.find(..)); repo3.save(d); mq.send(d.id); } g(dto) { repo2.save(mapper.fromDto(dto, repo1.find(..))); } SIMPLIFY PRODUCTION Collapse Middle-Man (useless method)
  14. 20 VictorRentea.ro §The Legacy Monolith ("Big ball of mud") has

    - Terrible complexity behind few entry points è Many Unit Tests are needed for the deeper corners of logic è §Tes(ng Microservices is different - Huge complexity in a single microservice = bad practice è Break it ! è More APIs, hiding a decent amount of complexity (🙏) è Possible to test more at the API level = Honeycomb Tes0ng Strategy
  15. 21 VictorRentea.ro Integra4on test one microservice endpoint end-to-end Integrated test

    the en0re ecosystem Implementa4on Detail Start up all microservices (in dockers / staging) Expensive, slow, flaky tests Required for business-criJcal flows (eg. checkout) Start up one microservice Use for: Every flow of the system Keep tests isolated without mocks Instan0ate several classes. Use for: naturally isolated parts of code with high complexity. Mocks are tolerated (but sJll avoided): Test components with clear roles using social unit tests Honeycomb Tes7ng Strategy Testcontainers 🐳 WireMock Contract Tests (Pact, SCC) DB ES Kafka ... API many tests on one entry point h,ps://engineering.atspo8fy.com/2018/01/tes8ng-of-microservices/ complexity decouple and test in isola/on
  16. 23 VictorRentea.ro Give you most Design Feedback 💎 More Complexity

    => BeDer Design Implementa%on Detail Tests 👍 MOCKS
  17. 27 VictorRentea.ro BAD HABIT Mock Roles, not Objects h1p://jmock.org/oopsla2004.pdf You

    implement a new feature > ((click in UI/postman)) > It works > 🎉 Oups!! I forgot about unit-tests 😱 ...then you write unit tests mocking all the dependencies of the prod code you wrote ↓ Few years later: "My tests are fragile and impede refactoring!" Contract-Driven Design Before mocking a dependency, clarify its responsibility = Changing an API you mocked is painful 😭
  18. 28 VictorRentea.ro write Social Unit Tests for "components" (groups of

    objects) with clear responsibili8es A B ✅ Internal refactoring won't break these tests Instead of solitary fine-grained unit tests mocking every dependency,
  19. 29 VictorRentea.ro "Unit Tes=ng means mocking all dependencies of a

    class" - common belief WRONG! "It's perfectly fine for unit tests to talk to databases and filesystems!"- Ian Cooper in his Talk Unit Tes$ng = ? Robust Unit Tes$ng requires iden$fying responsibili$es
  20. 31 VictorRentea.ro var bigObj = new BigObj(); bigObj.setA(a); bugObj.setB(b); prod.method(bigObj);

    Tests must create bigObj just to pass two inputs🫤 method(bigObj) MUTABLE DATA 😱 in 2023? using only 2 of the 15 fields in bigObj method(a, b) Precise Signatures prod.method(a, b); Also, simpler tests: Pass only necessary data to func$ons ✅ when(bigObj.getA()).thenReturn(a); ⛔ Don't Mock Ge_ers ⛔ ✅ Mock behavior, not data prod.method(new ABC(a, b, c)); method(abc) Parameter Object When tes$ng highly complex logic, introduce a ⛔ Don't return Mocks from Mocks⛔ when(bigObj.getA()).thenReturn(mockA); OMG! They sBck!
  21. 32 VictorRentea.ro 🏰 Constrained Objects = data structures that guard

    their internal consistency by throwing excep$ons, (eg required fields) Ø Mutable (eg Domain En$$es, Aggregates) Ø Immutable❤ (Value Objects)
  22. 33 VictorRentea.ro ❷ A group of classes has a clear

    role and manageable complexity ❶ A Constrained Object🏰 gets Large Object Mother Pa?ern TestData.aCustomer(): Customer (valid) coupling Break Domain En77es in separate Bounded Contexts packages > modules > microservices invoicing | shipping è Tes$ng the en$re group with a social unit-test✅ requires heavier data inputs * h1ps://mar/nfowler.com/bliki/ObjectMother.html (2006) Same Object Mother used in different ver+cals invoicing | shipping è Split Object Mother per ver4cal InvoicingTestData | ShippingTestData Crea$ng valid test data gets cumbersome CREEPY A large class shared by many tests Don't change it. Add methods, or tweak results ✅ TestData.charles(): Customer (a persona)
  23. 34 VictorRentea.ro Your complex logic directly uses External APIs or

    heavy libraries: Unit-tes$ng your logic requires understanding the seman$cs of: The External API: to populate/assert DTOs The Library: to mock it ... externalApi.call(apiDetails); ... dto.getStrangeField() ... Lib.use(mysteriousParam,...) Unit Tests should speak your Domain Model (mock the Adapter when tes/ng complex domain logic) Unit Tests are first-class ci$zens of your project #respect your tests Agnos7c Domain Isolate complex logic from the outside world ... clientAdapter.call(domainStuff) ... domainObject.getMyField() ... libAdapter.use(😊)
  24. 35 VictorRentea.ro applica3on / infra Value Object En0ty id Domain

    Service Domain Service agnos/c domain My DTOs External API External DTOs Client External Systems Applica0on Service Controller Repo Clean, Pragma/c Architecture at Devoxx Ukraine 2021 Curious for more? ↓ IAdapter Adapter Dependency Inversion ⛔ ⛔ Simplified Onion Architecture Interf W rapper f Dependency Inversion Ugly Invasive Library Domain complexity is kept inside for easier tes3ng
  25. 38 VictorRentea.ro class Big { f() { //complex g(); }

    g() { //complex } } Inside the same class, a complex func$on f() calls a complex g() g() is complex => unit-tested separately When tes$ng f(), can I avoid entering g()? Can I mock a local method call? class HighLevel { LowLevel low; f() {//complex low.g(); } } class LowLevel { g() {/*complex*/} } ↓ Par(al Mock (@Spy) Hard to maintain tests: Which method is real, which is mocked?🤯 Split by Layers of Abstrac7on (high-level policy vs low-level details) Tolerable tes+ng Legacy Code If splijng the class doesn't feel right, test f() + g() together with bigger tests
  26. 39 VictorRentea.ro class HighLevel { LowLevel low; f() {//complex low.g();

    } } Split by Layers of Abstrac7on = ver%cal split of a class class LowLevel { g() {/*complex*/} }
  27. 40 VictorRentea.ro class Wide { A a; B b; //+more

    dependencies f() {..a.a()..} g() {..b.b()..} } @ExtendWith(MockitoExtension) class WideTest { @Mock A a; @Mock B b; @InjectMocks Wide wide; // 5 tests for f() // 4 tests for g() } ↓ Split test class in ComplexFTest, ..G.. (rule: the before should be used by all tests) Complex methods in the same class use different sets of dependencies: class ComplexF { A a; f() { ..a.a().. } } class ComplexG { B b; g() { ..b.b().. } } @BeforeEach void fixture() { when(a.a()).then... when(b.b()).then... } Split Unrelated Complexity Later: what part of the before is used by my failed test? ** Mockito (since v2.0) throws UnnecessaryStubbingException if a when..then is not used by a @Test, when using MockitoExtension = Unmaintainable Tests 🤔 FIXTURE CREEP test setup DRY https://www.davidvlijmincx.com/posts/setting_the_strictness_for_mockito_mocks/ not used by these tests
  28. 41 VictorRentea.ro Split Unrelated Complexity class ComplexF { A a;

    f() { ..a.a().. } } class ComplexG { B b; g() { ..b.b().. } } horizontally ↔
  29. 42 VictorRentea.ro ver%cally ↕ Tests help us to break complexity

    horizontally ↔ When should we follow those hints? clear roles
  30. 46 VictorRentea.ro 1) Has no Side-Effects (doesn't change anything) INSERT,

    POST, send message, field changes, files 2) Returns Same Output for Same Inputs (no external source of data) GET, SELECT, current 8me, random, … Pure Func)on aka "Referen?al Transparency" Just compute a value 𝑒𝑔: 𝑀𝑎𝑡ℎ𝑒𝑚𝑎𝑡𝑖𝑐𝑎𝑙 𝐹𝑢𝑛𝑐𝑡𝑖𝑜𝑛𝑠: 𝑓(𝑥,𝑦)=𝑥^2+𝑦
  31. 47 VictorRentea.ro No Network or files No Changes to Data

    No .me/random Pure Func)ons immutable objects❤ (a simplified definiJon)
  32. 49 VictorRentea.ro a = repo1.findById(..) b = repo2.findById(..) c =

    api.call(..) 🤯complexity🤯 repo3.save(d); mq.send(d.id); Complex logic using many dependencies (eg: computePrice, applyDiscounts) Many tests using lots of mocks when(..).thenReturn(a); when(..).thenReturn(b); when(..).thenReturn(c); prod.complexAndCoupled(); verify(..).save(captor); d = captor.get(); assertThat(d)... verify(..).send(...); x15=😖 ✅ Easier to test, with less mocks d = prod.pure(a,b,c); assertThat(d)... Reduce Coupling of Complex Logic ✅ Easier to understand D pure(a,b,c) { 🤯complexity🤯 return d; }
  33. 50 © VictorRentea.ro a training by Complexity Func%onal Core Impera%ve

    Shell / Func%onal Core Segrega%on State Muta3on DB Impera%ve Shell API call Files Dependencies Complex Logic
  34. 51 © VictorRentea.ro a training by Func%onal Core Impera%ve Shell

    Impera%ve Shell / Func%onal Core Segrega%on Extract heaviest complexity as pure func5ons
  35. 52 VictorRentea.ro method(Mutable order, discounts) { ds.applyDiscounts(order, discounts); var price

    = cs.computePrice(order); return price; } ... but you use mutable objects Swapping two lines causes bugs in produc@on ❌ despite 4000 ✅ tests You have 4.000 unit tests, 100% test coverage 😲 👏 ↓ Paranoid Tes@ng (verify method call order) Immutable Objects method(Immutable order, d) { var discountedOrder = ds.applyDiscounts(order, d); var price = cs.computePrice(discountedOrder); return price; } TEMPORAL COUPLING Swapping the two lines ❌ breaks compila4on
  36. 54 VictorRentea.ro 1. Collapse Middle-Man vs "What am I tes,ng

    here?" Syndrome 2. Honeycomb Tes7ng Strategy: more Integra,on Tests than Fragile Unit Tests 3. Precise Signatures: less arguments 4. Dedicated Data Structures vs Creepy Object Mother 5. Agnos7c Domain vs using APIs or Libraries in complex logic 6. Split Complexity by Layers of Abstrac7on ↕ vs Par,al Mocks (@Spy) 7. Split Unrelated Complexity ↔ vs Fixture Creep (bloated setup) 8. Clarify Roles, Social Unit Tests vs blindly @Mock all dependencies 9. Decouple Complexity in Pure Func7ons vs Tests full of mocks 10.Immutable Objects vs Temporal Coupling Design Hints from Tests
  37. 56 VictorRentea.ro >meframe for developing your feature When do you

    start wri%ng tests? ✅ understand the problem => early ques3ons to biz ✅ early design feedback 💎 💎 💎 ✅ real test coverage => courage to refactor later BDD (.feature) too late TDD Good enough TDD!
  38. 58 VictorRentea.ro Unit Tes%ng Reading Guide (for later) 1] Classic

    TDD⭐⭐⭐ (mock-less) https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530 Mock Roles, not Objects ⭐⭐⭐: http://jmock.org/oopsla2004.pdf "Is TDD Dead?" https://martinfowler.com/articles/is-tdd-dead/ Why Most Unit Testing is Waste (James Coplien): https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf vs Integrated Tests are a Scam(J Brains): https://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam 2] London TDD⭐⭐⭐ (mockist) https://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627 3] Patterns⭐ https://www.amazon.com/Art-Unit-Testing-examples/dp/1617290890 4] https://www.amazon.com/xUnit-Test-Patterns-Refactoring-Code/dp/0131495054/ 5] (skip through) https://www.amazon.com/Unit-Testing-Principles-Practices-Patterns
  39. Your tests are trying to tell you something ... 10

    design hints you were missing Read the article: https://victorrentea.ro/blog/design-insights-from-unit-testing/ About the speaker: https://victorrentea.ro at è Stay connected. Join: