Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Road to 100% test coverage

Ruy Adorno
November 08, 2023

The Road to 100% test coverage

These are the slides for the workshop: "The Road to 100% test coverage". Presented at NodeConf EU 2023 in Kilkenny, Ireland.

More instructions on how to run the code generator / bootstrap can be found in the following GitHub repo: https://github.com/ruyadorno/create-100-workshop

Ruy Adorno

November 08, 2023
Tweet

More Decks by Ruy Adorno

Other Decks in Programming

Transcript

  1. NodeConf EU 2023
    100% test coverage
    The road to
    ruyadorno.com

    View full-size slide

  2. The road to 100% test coverage
    Ruy Adorno
    Senior Software Developer

    working at Google

    Node.js TSC

    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  3. TOC
    Setup the work environment for the workshop

    Understanding test coverage

    Techniques

    Questions / Consultation
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  4. Setup
    Install node 20 (or 18): https://nodejs.org/en/download

    Open your terminal, navigate to a suitable folder, e.g:
    cd ~/tmp/100-workshop

    Run the workshop generator:
    npm init @ruyadorno/100-workshop@latest

    Run npm install and tests:
    npm it
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  5. What’s in it?
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  6. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  7. Give it a try:
    List all LTS releases:
    node cli.js --filter=lts

    Last month’s releases:
    node cli.js --recent=month

    Only return the resulting count:
    node cli.js --count
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  8. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  9. What is a test
    A way to programatically confirm your program
    does what you think it does.

    If your tests are also a program, what validates that
    program?

    Surprise: It’s you, the developer!

    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  10. Test Coverage
    Tracking coverage is sort of a meta test

    It tracks what parts of the program are executed

    Gives you a hint of how much of a test it really is

    Definitely does not prove that either the source
    code OR the test code are good

    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  11. 100% Test Coverage
    It means you covered all lines, functions and
    statements

    Still not enough, 100% code coverage does NOT
    mean 100% of features tested

    Gives you super powers

    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com
    🦸

    View full-size slide

  12. Notice: This workshop has a cheat code available
    To quickly complete a step, it’s possible to use the
    same generator ran to create the boilerplate.

    Run the generator using the step option. In this
    example we’re using the generator to complete
    Step 1:
    npx @ruyadorno/create-100-workshop --step=1
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  13. Step 1
    Let’s add an initial test to the main index.js module!

    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  14. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com
    ./test/index.js

    ۪
    ۪ $ npx @ruyadorno/create-100-workshop --step=1
    ./src/index.js

    View full-size slide

  15. Step 2
    Let’s avoid the network request

    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  16. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com
    ./test/index.js

    ۪
    ۪ $ npx @ruyadorno/create-100-workshop —step=2

    View full-size slide

  17. Let’s visualize better the coverage results:
    Opens an html-based interface in your default browser:
    npm t -- --coverage-report=html
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  18. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  19. Step 3
    Let’s add tests for format-count.js
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  20. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com
    ./test/format-count.js

    ۪
    ۪ $ npx @ruyadorno/create-100-workshop —step=3

    View full-size slide

  21. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com
    🤩

    View full-size slide

  22. Step 4
    Let’s refactor and add tests for filter-recent.js
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  23. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com
    ./src/format-recent.js

    ۪
    ۪ $ npx @ruyadorno/create-100-workshop —step=4
    ./src/date.js

    View full-size slide

  24. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com
    ./test/format-recent.js

    ۪
    ۪ $ npx @ruyadorno/create-100-workshop —step=4

    View full-size slide

  25. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  26. Step 5
    Let’s add tests for filter-npm.js
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  27. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com
    ./test/filter-npm.js

    ۪
    ۪ $ npx @ruyadorno/create-100-workshop —step=5

    View full-size slide

  28. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  29. Step 6
    Let’s add tests for filter-booleans.js
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  30. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com
    ./test/filter-booleans.js

    ۪
    ۪ $ npx @ruyadorno/create-100-workshop —step=6

    View full-size slide

  31. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  32. Step 7
    Let’s add tests for the output.js module
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  33. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com
    ./test/output.js

    ۪
    ۪ $ npx @ruyadorno/create-100-workshop —step=7

    View full-size slide

  34. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  35. Step 8
    Let’s test retrieve-nodejs-versions.js
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  36. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com
    ./test/retrieve-nodejs-versions.js

    ۪
    ۪ $ npx @ruyadorno/create-100-workshop —step=8

    View full-size slide

  37. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  38. Step 9
    Let’s add the missing test to the index.js module
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  39. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com
    ./test/retrieve-nodejs-versions.js

    ۪
    ۪ $ npx @ruyadorno/create-100-workshop —step=9

    View full-size slide

  40. The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  41. Awesome Job!
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com
    👏

    View full-size slide

  42. Less than 100%
    If you can’t delete it, you should probably write tests
    for it so that you can maybe know what it does

    It means there are entire components of your
    application without tests

    What kind of program needs no test at all?
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  43. Less than 100%
    Would you rely on a module with no tests?

    A 95% test-covered program is two parts: the
    covered part and the uncovered part

    The uncovered part is a module with no tests
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  44. Bene
    fi
    ts
    Testable code === maintainable code

    Full coverage forces you to look in dark corners

    Investigation process tends to uncover bugs

    Let you focus on what you’re good at while the
    computer keeps track of everything else you’ve
    already checked
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  45. Human aspect
    Going from 98% to 96% looks and sounds ok

    Going from 100% to 99.5% is a huge loss!

    It feels great to see all these rows of 100%
    covered modules

    It’s addictive 😁 and will trick you into being a
    better developer!
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  46. Extra costs
    Full coverage adds up-front to development cost

    In return it reduces the incidence of bugs, and the
    costs of changing a program

    Less technical debt!
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  47. Extra costs
    Full test coverage will not prevent all bugs, what it
    does is gives you the power to fix these bugs more
    efficiently

    What about TypeScript and eslint preventing most
    errors already?

    Soon after you wrote types you no long knows
    what the code is actually supposed to do
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  48. Bad tests
    People will write bad tests with less than 100%
    coverage too

    The solution is to write better tests, and more
    easily tested components, not to accept untested
    code

    Only testing the happy path is the equivalent of
    “works on my machine”
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  49. Levelling Up
    Limit coverage values to only account for a test
    written for a specific module

    Avoiding accidental coverage, e.g: a base class
    method from module A gets coverage when run from
    module B tests

    tap supports it out-of-the-box with its coverage-map
    config option
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  50. Techniques
    Tests with long assertion output that have to be
    updated on every change? Use snapshots for that,
    save them in git, review the diff.

    Have to mock the whole world to hit a buried path?
    Refactor it into a discrete submodule, and only mock
    that module once. You’re now writing better code.
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  51. Techniques
    Full coverage isn’t about writing code with more
    tests, it’s about writing code that is more testable.

    Monkey-patching and other “bad” software
    development practices are usually totally fine in
    tests!
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  52. Techniques
    Things that really cannot be tested for some reason?
    Use ignore comments.

    This is an explicit (and well-documented, it’s going to
    live there with your code) way of opting-out of
    coverage for a very valid reason.
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  53. End of the workshop
    Questions? Consultation time.
    The road to 100% test coverage
    NodeConf EU 2023 ruyadorno.com

    View full-size slide

  54. The road to 100% test coverage
    Isaac Z. Schlueter
    node-tap author & author of the original
    “99% is not enough” talk this workshop is
    based from.

    NodeConf EU 2023 ruyadorno.com
    Special thanks to:

    View full-size slide

  55. The road to 100% test coverage
    Ruy Adorno
    Linkedin: /in/ruyadorno

    @ruyadorno

    @[email protected]

    @ruyadorno.bsky.social

    NodeConf EU 2023 ruyadorno.com
    Thanks! Find me at:

    View full-size slide