Upgrade to Pro — share decks privately, control downloads, hide ads and more …

He has 100% code coverage and writes FEWER test...

He has 100% code coverage and writes FEWER tests! [SEE HOW]

Sebastian Buczyński

June 30, 2023
Tweet

Transcript

  1. He has 100% code coverage and writes FEWER tests! Sebastian

    Buczyński @ Pycon PL 30.06.2023 [SEE HOW]
  2. Automated tests >>> tedious manual checks def test_dialing_another_phone_makes_it_ring() -> None:

    phones.register("Alice", "111") phones.register("Bob", "222") phones.of("Alice").dial("Bob") assert phones.of("Bob").is_ringing()
  3. def determine_queue_position(patient: Patient, queue: Queue) -> int: position = len(queue)

    if patient.is_pregnant or patient.is_regular_blood_donor: position = 0 return position
  4. def determine_queue_position(patient: Patient, queue: Queue) -> int: position = len(queue)

    if patient.is_pregnant or patient.is_regular_blood_donor: position = 0 return position
  5. def determine_queue_position(patient: Patient, queue: Queue) -> int: position = len(queue)

    if patient.is_pregnant or patient.is_regular_blood_donor: position = 0 return position
  6. def test_determine_queue_position() -> None: queue = Queue() patient = Patient(is_pregnant=False,

    is_regular_blood_donor=False) position = determine_queue_position(patient, queue) assert position == 0
  7. def test_determine_queue_position() -> None: queue = Queue() patient = Patient(is_pregnant=False,

    is_regular_blood_donor=False) position = determine_queue_position(patient, queue) assert position == 0
  8. def test_determine_queue_position() -> None: queue = Queue() patient = Patient(is_pregnant=False,

    is_regular_blood_donor=False) position = determine_queue_position(patient, queue) assert position == 0
  9. test_doctor_queue_1.py . [100%] ---------- coverage: platform darwin, python 3.11.0-final-0 ----------

    Name Stmts Miss Cover Missing --------------------------------------------------------- doctor_queue/ __ init __ .py 0 0 100% doctor_queue/determine.py 11 1 91% 18 --------------------------------------------------------- TOTAL 11 1 91%
  10. • Trainer / Consultant @ Bottega IT Minds • TL

    @ Sauce Labs • Evangelist of software engineering in Python • Substack - pythoneer.substack.com • Fan of Bartosz Walaszek • Apprentice of alternative coffee brewing methods whoami
  11. pytest test_doctor_queue.py -- cov doctor_queue/ —cov - report=term - missing

    test_doctor_queue_1.py . [100%] ---------- coverage: platform darwin, python 3.11.0-final-0 ---------- Name Stmts Miss Cover Missing --------------------------------------------------------- doctor_queue/ __ init __ .py 0 0 100% doctor_queue/determine.py 11 1 91% 18 --------------------------------------------------------- TOTAL 11 1 91% Top 10 Photos Taken Before Tragedy Generated by Midjourney
  12. class SomeClass: ... @property def some_function(self) -> int: return 42

    # <-- you don't have coverage for this line, bro „Let’s rewrite this project!”
  13. Code coverage is not that great def test_pregnancy_means_accessing_doctor_without_having_to_wait() : queue

    = Queue() patient = Patient(is_pregnant=True, is_regular_blood_donor=False) queue_position = determine_queue_position(patient, queue) assert queue_position == 0
  14. Code coverage is not that great def test_pregnancy_means_accessing_doctor_without_having_to_wait() : queue

    = Queue() patient = Patient(is_pregnant=True, is_regular_blood_donor=False) queue_position = determine_queue_position(patient, queue) assert queue_position == 0
  15. Hints during development process (parts of code that are not

    covered are something to take care about next)
  16. 🚫 Horizontal slicing 🚫 1. Integrate with vendor’s API 2.

    Create Backend service with API to browse offers and buy stuff 3. Frontend
  17. ✅ Vertical slices ✅ 1. PoC of integration with one

    of vendor’s API 2. Browsing offers from one vendor 3. Ability to make a purchase on one vendor 4. Adding 2nd vendor with refactoring 5. Adding more vendors
  18. Observable results in tests What to assert? • Where can

    a user see some change after their action? • Where other types of users can see some effect? • How this action affects other features? • What a user can now do which they couldn’t do before? • What a user can no longer do which they were able to do before?
  19. Development process with USEFUL 100% code coverage • Step #1

    - Pick up a task and ensure it can be veri f ied • Step #2 - Come up with acceptance testing scenario(s) • Step #3 - Write a test for this scenario (or at least empty test) • Step #4 - Implement the feature • Step #5 - Check code coverage • Repeat steps 2-5 until satis f ied
  20. Example #1 A customer can store their payment card… and

    later retrieve it def test_customer_can_read_stored_payment_card_details() -> None: pass
  21. def test_customer_can_read_stored_payment_card_details( authorized_client: TestClient, ) - > None: add_response =

    authorized_client.post( "/payment - cards", json={ "card_number": "4111 1111 1111 1111", "names": "John Doe", "cvc": "123", }, )
  22. def test_customer_can_read_stored_payment_card_details( authorized_client: TestClient, ) - > None: add_response =

    authorized_client.post( "/payment - cards", json={ "card_number": "4111 1111 1111 1111", "names": "John Doe", "cvc": "123", }, ) assert add_response.status_code == 201
  23. def test_customer_can_read_stored_payment_card_details( authorized_client: TestClient, ) - > None: add_response =

    authorized_client.post( "/payment - cards", json={ "card_number": "4111 1111 1111 1111", "names": "John Doe", "cvc": "123", }, ) assert add_response.status_code == 201 get_response = authorized_client.get("/payment - cards") assert get_response.status_code == 200 assert get_response.json() == [ { "card_number": "4111 **** **** 1111", "names": "John Doe", "cvc": " ** * ", } ]
  24. We check coverage… tests/acceptance/test_storing_payment_card.py . [100%] ---------- coverage: platform darwin,

    python 3.11.0-final-0 ---------- Name Stmts Miss Branch BrPart Cover ------------------------------------------------------------------ subscriptions/ __ init __ .py 0 0 0 0 100% subscriptions/anonymization.py 4 0 0 0 100% subscriptions/api.py 31 0 8 0 100% subscriptions/db.py 7 0 2 0 100% subscriptions/models.py 9 0 0 0 100% ------------------------------------------------------------------ TOTAL 51 0 10 0 100%
  25. We look for more test scenarios …and repeat def test_initially_customer_has_no_saved_payment_cards()

    -> None: pass def test_customer_can_read_only_payment_cards_they_created() -> None: pass
  26. Digression: veri f ication methods There are only 4 •

    Returned result (from a function / a method) • Raised exception (or lack of it) • State veri f ication - we call one method / endpoint, then we check result using other method / endpoint • Interaction - Mocks • e.g. assert_called_once_with
  27. @app.post("/payment - cards - in - external - system") def

    store_payment_card_in_external_system( payload: PaymentCardPayload, user_id: int = Depends(user_id), payments_gateway: PaymentsGateway = Depends(), ) -> Response: httpx.post("https: // example.tech/payment - cards", json={ "card_number": payload.card_number, "names": payload.names, "cvc": payload.cvc, "user_id": user_id, }) return Response(status_code=201)
  28. class PaymentsGateway: def store_payment_card(self, ... ) -> None: httpx.post("https: //

    example.tech/payment - cards", json={ "card_number": dto.card_number, "names": dto.names, "cvc": dto.cvc, "user_id": dto.user_id, })
  29. @define(frozen=True, repr=False) class PaymentCardDto: user_id: int card_number: str names: str

    cvc: str class PaymentsGateway: def store_payment_card(self, dto: PaymentCardDto) -> None: httpx.post("https: // example.tech/payment - cards", json={ "card_number": dto.card_number, "names": dto.names, "cvc": dto.cvc, "user_id": dto.user_id, })
  30. def test_payment_card_is_stored_in_external_system( authorized_client: TestClient, user_id: int, when: Any, ) ->

    None: when(PaymentsGateway).store_payment_card( ... ).thenReturn(None) add_response = authorized_client.post( ... ) assert add_response.status_code == 201 verify(PaymentsGateway, times=1).store_payment_card( PaymentCardDto( user_id=1, card_number="4000-0000-0000-0002", names="Janine Doe", cvc="321", ) )
  31. tests/acceptance/test_storing_payment_card.py ... [ 75%] tests/acceptance/test_storing_payment_card_with_external_system.py . [100%] ---------- coverage: platform

    darwin, python 3.11.0-final-0 ---------- Name Stmts Miss Branch BrPart Cover --------------------------------------------------------------------- subscriptions/ __ init __ .py 0 0 0 0 100% subscriptions/anonymization.py 4 0 0 0 100% subscriptions/api.py 38 0 10 0 100% subscriptions/db.py 7 0 2 0 100% subscriptions/models.py 9 0 0 0 100% subscriptions/payments_gateway.py 11 1 2 0 92% --------------------------------------------------------------------- TOTAL 69 1 14 0 99%
  32. Is this safe? With strict mypy settings - yes. @app.post("/payment

    - cards - in - external - system") def store_payment_card_in_external_system( ... ) -> Response: dto = PaymentCardDto( user_id=user_id, card_number=payload.card_number, names=payload.names, cvc=payload.cvc, ) payments_gateway.store_payment_card(dto) return Response(status_code=201)
  33. More ways to use Gateway Pattern • In tests we

    can emulate behaviour of the external system (fake) • We translate errors we want to handle explicitly for example to exceptions
  34. Testing strategy with 100% code coverage • Start from acceptance

    tests • Add more lower-level tests if necessary (fractal testing) • Test(s) end-to-end / synthetic monitoring etc • # pragma: no cover where it makes sense (e.g. abstract methods)
  35. My acceptance tests have few hundred lines :( • Your

    architecture doesn’t support testing • You need more modularization (think smaller components)