$30 off During Our Annual Pro Sale. View Details »

Are Your Tests Slowing You Down?

Trisha Gee
October 03, 2023

Are Your Tests Slowing You Down?

Testing is a Good Thing, right? Especially automated testing. But "Good things come to those who wait" is not something that's going to appeal to the busy developer. You want results, and you want them now. You're in The Zone working on a problem, and the last thing you want is to break your flow wrestling with your testing framework or waiting for the tests to finish running.

More code means more tests. More coverage means more tests. More tests mean more time. Time that you want to spend being productive, creative, innovative. How can you balance the need for quality with the need for speed?

In this talk, Trisha will identify issues that slow down developers when writing, running and debugging tests, and look at tools that can help developers with each of these problems. There will be live coding, analysis of social media poll results, an overview of solutions in this space, "best practice" recommendations, and machine learning will be mentioned at some point.

Trisha Gee

October 03, 2023
Tweet

More Decks by Trisha Gee

Other Decks in Technology

Transcript

  1. Are Your Tests Slowing You Down?
    Trisha Gee

    View Slide

  2. https://trishagee.com/books/

    View Slide

  3. Gradle is Pioneering DPE
    Developer Productivity Engineering
    is a software development practice
    to maximise developer productivity
    and happiness.

    View Slide

  4. View Slide

  5. Tests Make You More Productive?
    Tests Make You More Productive

    View Slide

  6. View Slide

  7. View Slide

  8. View Slide

  9. View Slide

  10. So it’s OK not to write tests, right?

    View Slide

  11. “Testing is Xanax for programmers, it’s an anxiety
    reducing technique.”
    https://youtu.be/guycIP56YeY


    Kent Beck

    View Slide

  12. Caveats

    View Slide

  13. RATED 6+


    because apparently 5-year-olds fall asleep during the presentation


    Some flashing GIF sequences may affect photosensitive viewers


    This program contains product placement

    View Slide

  14. “Nothing is ever really lost to us as long as we
    remember it.”
    L.M. Montgomery


    View Slide

  15. What slows down your testing?

    View Slide

  16. 1. Writing Tests Takes Time

    View Slide

  17. “Trisha, you’re actually a tester”
    Liz Keogh (@lunivore)


    View Slide

  18. Use Your IDE

    View Slide

  19. Buy My Book!

    View Slide

  20. View Slide

  21. AI Will Make You Super Productive

    View Slide

  22. View Slide

  23. View Slide

  24. Beware, AI lies to you

    View Slide

  25. Bring a friend

    View Slide

  26. “We recommend that the developers, who are good at
    writing efficient, maintainable code, work together
    with the testers, who are good at specifying test cases,
    to automate tests.”
    Agile Testing Condensed: A Brief Introduction


    Lisa Crispin, Janet Gregory


    View Slide

  27. https://bit.ly/3RG85vc

    View Slide

  28. “Creativity is a wild mind and a disciplined eye.”
    Dorothy Parker


    View Slide

  29. Invest the time in writing to save time troubleshooting

    View Slide

  30. 2. Troubleshooting Takes Time

    View Slide

  31. What is this test anyway?

    View Slide

  32. View Slide

  33. View Slide

  34. View Slide

  35. Is it the test that’s wrong, or
    production code?

    View Slide

  36. Let me see…

    View Slide

  37. Let me see…

    View Slide

  38. View Slide

  39. Aim for fast feedback and easy
    reproduction

    View Slide

  40. View Slide

  41. “99.9% of the time the best way to troubleshoot a test
    is to re-write it so it’s smaller and faster.”
    Israel Boza Rodriguez (@IsraKaos)


    View Slide

  42. Flaky Tests

    View Slide

  43. I have made the decision to trust you
    Body text


    View Slide

  44. I have made the decision to trust you
    Body text


    View Slide

  45. View Slide

  46. View Slide

  47. Fix your flaky tests!!!

    View Slide

  48. https://youtu.be/vHBzZHE4tJ0

    View Slide

  49. 3. Running Tests Takes Time

    View Slide

  50. How do you speed up your tests?

    View Slide

  51. Use a build cache

    View Slide

  52. Very effective for multi-module projects

    View Slide

  53. No cache: Full build required

    View Slide

  54. Build cache: Changing a leaf module

    View Slide

  55. Build cache: Changing a module’s implementation

    View Slide

  56. Build cache: Changing core implementation

    View Slide

  57. Remote Build Cache

    View Slide

  58. Run tests in parallel (local)

    View Slide

  59. Run tests in parallel (CI)

    View Slide

  60. View Slide

  61. Test Distribution

    View Slide

  62. View Slide

  63. View Slide

  64. Test Distribution Results
    ‑ ~50%
    ‑ ~50%
    ‑ ~50%
    Doubling the number of executors cuts build time in half

    View Slide

  65. Netflix reduced a 62-minute test cycle time down to just under 5 minutes!

    View Slide

  66. Let’s use AI!

    View Slide

  67. “The test selection procedure selects fewer than a third


    of the tests that would be selected on the basis of build


    dependencies”
    Predictive Test Selection - Meta Research


    https://research.facebook.com/publications/predictive-test-selection/


    Mateusz Machalica, Alex Samylkin, Meredith Porth, Satish Chandra


    View Slide

  68. Predictive Test Selection time savings

    View Slide

  69. But Trisha, I just have So Many Tests!

    View Slide

  70. “Yes, I regularly delete tests that no longer add unique
    value.”
    Angie Jones (@techgirl1908)

    View Slide

  71. View Slide

  72. View Slide

  73. View Slide

  74. Like any performance problem,


    focus on the worst-performers first

    View Slide

  75. View Slide

  76. “Once things like databases become a hindrance,
    optimise the testing loop, mock things out to run things
    fast. You can run locally and you don't have all the same
    infra dependencies as someone else.”
    On DevEx, PaaS, and Testing in Production https://bit.ly/3ZA0uQM


    Kelsey Hightower

    View Slide

  77. Look out for expensive setup and tear
    down

    View Slide

  78. View Slide

  79. View Slide

  80. Respect your test code

    View Slide

  81. View Slide

  82. View Slide

  83. View Slide

  84. ⬢ Separation of concerns
    ⬢ Encapsulation
    ⬢ SOLID
    ⬢ DRY
    ⬢ Descriptive naming
    ⬢ Modularisation
    ⬢ Designing for performance

    View Slide

  85. Tension between these ideals

    View Slide

  86. “Tidying is software design addressing you, your
    relationship to your code, and ultimately your
    relationship with yourself…. Tidying is geek self-care.”
    Tidy First?


    Kent Beck


    View Slide

  87. Summary

    View Slide

  88. Automate as much as possible

    View Slide

  89. There are solutions out there designed for exactly this problem

    View Slide

  90. There is a time investment in writing tests,


    because there should be thinking involved

    View Slide

  91. “Your best and wisest refuge from all troubles is in your
    science.”
    Ada Lovelace


    View Slide

  92. Thank you!

    View Slide