Contract testing with Java

If you ever had problems testing and managing dependencies in a microservice environment, or just wonder where contract tests fit on the testing pyramid, join this talk for a discussion on contract tests from the basics to the limits of the paradigm!
In this talk we will cover the basics of testing, the benefits contract tests offer for both synchronous and asynchronous communication and the limits of these tests. We will briefly cover the two most prominent contract testing frameworks and we will discuss the difference between them.

Transcript

2. Do it PERFECT. 2 Agenda 1. Why do we need testing? 2. What is contract testing? 3. How can we use contract tests? 4. When do we reach the limits of contract tests?
3. Do it PERFECT. 3 Hi • Damjan Gjurovski • Software Engineer • Java
4. Do it PERFECT. 4 Hi • Damjan Gjurovski • Software Engineer • Java • Kotlin • Typescript • React
5. Do it PERFECT. 5 Hi • Damjan Gjurovski • Software Engineer • Java • Kotlin • Typescript • React • DevOps
6. Do it PERFECT. 6 Hi • Damjan Gjurovski • Software Engineer • Java • Kotlin • Typescript • React • DevOps • Platform Engineer
7. Do it PERFECT. 7 Hi • Damjan Gjurovski • Software Engineer • Java • Kotlin • Typescript • React • DevOps • Platform Engineer • CTO @ Posedio
8. Do it PERFECT. 8 Done things, many that require tests • Damjan Gjurovski • Software Engineer • Java • Kotlin • Typescript • React • DevOps • Platform Engineer • CTO @ Posedio
9. Do it PERFECT. 9 Seen things, many that shouldn‘t be seen • Damjan Gjurovski • Software Engineer • Java • Kotlin • Typescript • React • DevOps • Platform Engineer • CTO @ Posedio
10. Do it PERFECT. 10 01 WHY DO WE NEED TESTING?
11. Do it PERFECT. 11 # • Tell the machine what to do • After having been told what the machine should do • Machine always does what it’s told, so: Software development
12. Do it PERFECT. 12 # • Tell the machine what to do • After having been told what the machine should do • Machine always does what it’s told, so: • Did we understand what needs to be done? • Did we correctly translate it? • Does the user know what to tell the machine to get it to cooperate? Software development
13. Do it PERFECT. 13 # • Put ourselves in the position of a user • Ask the machine what it would do if we were to do something • Given input A (and state S), what happens? • Given input A (and state S) I expect that output B (and state M) • The earlier we find a problem, the easier it is to fix (common sense, but also scientific evidence) Testing
14. Do it PERFECT. 14 # • We can do unit tests without help • We test the code we wrote • It’s fast • We might need help from colleagues for integration tests, or we might need mocks, or we might need a test environment • We test other people's code • It’s slow(er) • We need a full environment • We test the system • It’s not even worth talking about speed The pyramid https://martinfowler.com/bliki/TestPyramid.html
15. Do it PERFECT. 15 02 WHAT IS CONTRACT TESTING?
16. Do it PERFECT. 16 # • Tell the machine what to do • After having been told what the machine should do • After another developer told the machine what it should do • Expecting that everyone is on the same page • We can no longer rely on our understanding of the requirements, but must ensure we have a shared understanding with another team Distributed systems
17. Do it PERFECT. 17 # • Consumer defines the API • Consumer can test against a mock that implements the API • Provider can verify the implementation of the API Contract test https://microsoft.github.io/code-with-engineering-playbook/automated-testing/cdc-testing/
18. Do it PERFECT. 18 # • Decouple consumers and producers • Provide a schema for every message • What happens if the consumer doesn’t know what to do with the schema? • Coordinated deployments What about event driven systems?
19. Do it PERFECT. 19 # • Decouple consumers and producers • Provide a schema for every message • What happens if the consumer doesn’t know what to do with the schema? • Coordinated deployments What about event driven systems?
20. Do it PERFECT. 20 # • A schema defines what the data should look like and potentially what interactions are allowed • Syntax • Abstract • A contract defines how two systems can communicate • Semantics • Concrete • Contract = schema + examples + other nice things Contract test vs. Schema https://pactflow.io/blog/schemas-are-not-contracts/
21. Do it PERFECT. 21 03 HOW CAN WE USE CONTRACT TESTS?
22. Do it PERFECT. 22 # • Provide a mock (Test Double) • Have that mock respond the same as the remote service • Check if the response is the same from time to time • Should we check when we make a change to the consumer or the producer? Test double https://martinfowler.com/bliki/ContractTest.html
23. Do it PERFECT. 23 # • Consumer creates contract • Pull request to the contract repository • Review by the producer • Consumer generates a stub based on the contract • Tests execute against the stub • Producer generates a stub against the contract • Tests execute against the stub • Changes to the producer break the build (before breaking the consumer) Contract testing workflow https://softwaremill.com/testing-microservices-contract-tests/
24. Do it PERFECT. 24 # Pact for Java - REST https://www.baeldung.com/pact-junit-consumer-driven-contracts
25. Do it PERFECT. 25 # Pact for Java - REST https://www.baeldung.com/pact-junit-consumer-driven-contracts • Extend the “Integration tests” with a mechanism to ensure the two systems are in sync, by • Autogenerating a mock based on specification • Testing both sides for compatibility • Decoupling business logic tests (functional tests) from communication tests (contract tests) • We can still test the business logic on our end!
26. Do it PERFECT. 26 # Pact for Java - Consumer https://www.baeldung.com/pact-junit-consumer-driven-contracts
27. Do it PERFECT. 27 # Pact for Java - Consumer https://www.baeldung.com/pact-junit-consumer-driven-contracts
28. Do it PERFECT. 28 # Pact for Java - Producer https://www.baeldung.com/pact-junit-consumer-driven-contracts
29. Do it PERFECT. 29 # • Setup is similar • Consumer defines what message is expected • Contract is published • Consumer uses autogenerated mock to “consume” a message • Producer uses autogenerated mock to check if a published message conforms to the contract • Kafka schema is not (!) checked • But usage of a schema is checked Pact for Java - Kafka https://docs.pact.io/recipes/kafka
30. Do it PERFECT. 30 # Pact for Java – Kafka consumer https://docs.pact.io/recipes/kafka
31. Do it PERFECT. 31 # Pact for Java – Kafka consumer https://docs.pact.io/recipes/kafka
32. Do it PERFECT. 32 # Pact for Java – Kafka producer https://docs.pact.io/recipes/kafka
33. Do it PERFECT. 33 # Pact for Java – Kafka producer https://docs.pact.io/recipes/kafka
34. Do it PERFECT. 34 # • On the consumer side: • We want to mock a service so we can test things quickly • Mocking means we ensure that given an input it will return an output • We are actually interested in testing the consuming system, not the mocked/producing system • But to do that we need to be sure that what we have mocked is correct (the real system will indeed return a given output for a given input) What do we actually want to verify https://docs.pact.io/recipes/kafka • On the producer side: • We want to make sure if we make changes, we do not break the expectations of our consumers • We want to be sure we share the same understanding of the system as our consumers • We are not actually testing our code or our logic, just ensuring that changes are properly communicated (breaking the build would force us to communicate)
35. Do it PERFECT. 35 # • Very similar, but: • A bit more verbose • No broker for contracts • Means you need to ensure contracts are trusted • Binds you into Spring • Although probably integrates better if you are already using Spring Spring Cloud Contract https://docs.spring.io/spring-cloud-contract/reference/getting-started/introducing-spring-cloud-contract.html
36. Do it PERFECT. 36 04 THE LIMITS OF CONTRACT TESTS
37. Do it PERFECT. 37 # Don’t take it too far https://pactflow.io/blog/five-reasons-why-your-contract-testing-initiative-could-fail-and-how-to-avoid-them/ • Contract tests are meant to check the communication part between consumer and provider, not the full business logic • Use functional tests for this • For example, you can have one positive contract tests and one negative contract test for fraud detection, even if there are 100 cases when fraud detection reports a failure
38. Do it PERFECT. 38 # Communication is key • Contract tests help in catching miscommunication • They won't fix the problem, only make it visible • Without the producing team running the test, nothing will work • Producing team has little incentive to run the tests
39. Do it PERFECT. 39 # It’s still a test https://medium.com/contract-testing/contract-testing-navigating-through-common-pitfalls-e2a78c44a1da • Be wary of false positives and false negatives • Maybe the test does not check all fields in the contract • Maybe the test checks more fields than specified in the contract
40. Do it PERFECT. 40 # Versioning + Dependencies • Contract tests show us when a contract changes • We need to figure out how to version changes and manage dependencies • No easy answer, but maybe a subject of another talk J
41. Do it PERFECT. 41 THANK YOU!