QA&Test Bilbao - API Testing for embedded projects with Testcontainers and Wiremock

Embedded software engineers can have nice things, too! Overview of Testcontainers and WireMock usage for C/C++ projects

Transcript

1. Fake it until you `make it` API Integration testing of embedded projects with Testcontainers and WireMock Oleg Nenashev, WireMock Inc QA&Test 2023 Bilbao
2. `make it` ?
3. Integration testing? 3 • Multiple technology stacks and toolchains • Many teams involved • Sacred knowledge • [Almost] always too late
4. Outline • Intro to API integration testing • Testcontainers and WireMock • Using them together • And examples! C/C++, Golang, Rust 4 bit.ly/qatest2023_wiremock Disclaimer: This is a talk about “WireMock AND Testcontainers”, not “VERSUS” ir “OR”. Your mileage may vary These slides:
5. > whoami @oleg_nenashev oleg-nenashev Dr. Nenashev / Mr. Jenkins Developer tools hacker Community builder & DevRel consultant #RussiansAgainstPutin #StandWithUkraine
6. @oleg_nenashev oleg-nenashev > whoami --automotive --embedded --hardware 6 CloudBees Embedded & Automotive Day in Stuttgart LibreCores CI
7. Do you already use container technologies in test automation? 7
8. 8 Also: 2016 - Pini Reznik, Containers and test automation & More & More
9. Why containers? • Configuration Management • Fast provisioning • Clean environments • Disposability • Clustering
10. Docker in Embedded Prototypes, Dev Tools Test Infrastructures, CI/CD Docker Swarm Cluster on Raspberry Pi
11. Even simulator is not always an option http://blogs.synopsys.com/breakingthethreelaws 2014/04/the-secret-ninja-fu-for-higher-performa e-prototype-operation/ HAPS S96 Guess the price… Your device prototype is here
12. Docker • Popular container engine • Developer-friendly • Huge ecosystem • DockerHub and Container Registries • Docker Compose – multi-container apps • Universal image format (OCI) https://www.docker.com/
13. Do your projects have APIs? 13
14. APIs are essential for Integration Testing 14 GROWTH IN WEB APIS SINCE 2005 Over 90% of developers use APIs. Skyrocket growth of APIs JANUARY 2006 JANUARY 2008 JANUARY 2010 JANUARY 2012 JANUARY 2014 JANUARY 2016 JANUARY 2018 MONTH The growth over time of the Programmable Web API API directory to more than 22,000 entries 22000 20000 18000 16000 14000 12000 10000 8000 6000 4000 2000 0 TOTAL API COUNT Programmable Web * Gartner Hype Cycle for APIs, 2022
15. APIs
16. 16 Source: https://medium.com/10-minutes-qa-story/api-testing-from-scratch-api-anatomy-f220856ce9b3
17. APIs are essential for Integration and Integration Testing 17
18. Ways to do API integration testing 1. Testing against Production/Staging servers 2. Testing against a simplified/containerized instance (e.g. Testcontainers) 3. Mocking at the API provider level (e.g. WireMock) 4. Mocking at the code level 18 Slow Fast Accu- rate Nope
19. Ways to do API integration testing 1. Testing against Production/Staging servers 2. Testing against a simplified/containerized instance (e.g. Testcontainers) 3. Mocking at the API provider level (e.g. WireMock) 4. Mocking at the code level 19 Slow Fast Nope Accu- rate
20. 20 Build Unit tests Publish Reports Integration tests Publish Reports WAS
21. 21 Build Unit tests Publish Reports Integration tests Publish Reports NOW “Shift Left” Fast integrations tests are critical
22. 22 Staging & QA Tests Build Integration tests Build Tests Gaining confidence in you software
23. 23 Shifting ITs left with Testcontainers and WireMock
24. Ways to do API integration testing 1. Testing against Production/Staging servers 2. Testing against a simplified/containerized instance (e.g. Testcontainers) 3. Mocking at the API provider level (e.g. WireMock) 4. Mocking at the code level 24 Slow Fast Accu- rate Nope
26. 26 SDKs C/C++ testcontainers.org
28. Example - JUnit 5 28
29. Containers are not always slow! • On-demand image build • Caching Docker image builds • Suspending containers between tests • Graceful termination ◦ github.com/testcontainers/moby-ryuk 29 testcontainers.org
30. 30 https://wiremock.org
31. Ways to do API integration testing 1. Testing against Production/Staging servers 2. Testing against a simplified/containerized instance (e.g. Testcontainers) 3. Mocking at the API provider level (e.g. WireMock) 4. Mocking at the code level 31 Slow Fast Accu- rate Nope
32. 32 WireMock 101 WireMock is a tool for building mock APIs Created in 2011 as a Java test library and CLI Available beyond Java, inc Rust or Golang You can can: • Create stable development environments • Isolate yourself from flakey 3rd parties • Simulate APIs that don't exist yet wiremock.org
33. 33 wiremock.org/docs/stubbing IF (request_url) THEN (response) * * it gets MUCH more complex WireMock Config JSON: Client library response request Client App Mock API Server HTTP/2
34. Breaking down JSON files Using WireMock Cloud app.wiremock.cloud 34
35. WireMock in Java 35 wiremock.org/docs/stubbing
36. WireMock Ecosystem & Features 36 wiremock.org/docs
37. WireMock Ecosystem & Features 37 wiremock.org/docs
38. Example - WireMock for Rust 38 docs.rs/wiremock/latest/wiremock LukeMathWalker/wiremock-rs Note: WireMock for Rust is maintained outside the WireMock GitHub organization
39. Comparison [1/2] 1. Testing against Production/Staging servers 2. Testing against a simplified/containerized instance (e.g. Testcontainers) 3. Mocking at the API provider level (e.g. WireMock) 4. Mocking at the code level 39 Slow Fast Nope Accu- rate
41. 41 WireMock has an official Testcontainers module! testcontainers.com/modules/wiremock
42. Example - JUnit 5 42
43. WireMock as a Proxy 43 Tests Real API Provider OR • Fault injection • Recording • Protocol Verification
44. 44 How about embedded?
45. IEEE Spectrum Research among members 45 https://spectrum.ieee.org/top-progra mming-languages-2021
46. 46 jetbrains.com/lp/ devecosystem-2021/embedded/
47. 47 Supported by WireMock and Testcontainers C/C++ Shared Lib Interface
48. Testcontainers for C/C++ 48 Started by the WireMock Team in September 2023 We have Embedded/Automotive customers and users who need it Based on Testcontainers or Go WireMock module github.com/testcontainers/testcontainers-c
49. Example - WireMock with C code 49 github.com/testcontainers/testcontain ers-c/tree/main/examples/wiremock • Testcontainers C is an experimental project • Feedback is welcome! • See more examples in the repository: CppUnit, OpenSCADA, etc.
50. 50 WireMock for Google Test / C++ github.com/testcontainers/te stcontainers-c/tree/main/de mo/google-test
51. And then a test! 51 github.com/testcontainers/testcontainers-c/tree/main/demo/google-test
52. 52 Demo time! Integration Testing With WireMock
53. Takeaways 53 • Shift left your integration testing • Mocking APIs and services, reducing external dependencies, containers • There are tools for that, including WireMock and Testcontainers • WireMock and Testcontainers co-exist well • They work for embedded use-cases!
54. Choosing a right tool * ** * Your mileage may vary ** You will often need both and that’s OK! Need database or complex dynamic environment? Yes Server implementation is ready? Is it containerized and lightweight? Can you [easily] transfer the system to the required state in test? No No No No Yes Yes Yes 54
55. Get Started wiremock.org/docs/solutions/embedded 55
56. THANK YOU Contacts: E-mail: oleg@wiremock.org GitHub: oleg-nenashev Twitter: @oleg_nenashev QUESTIONS? 56
57. WireMock Cloud by WireMock Inc. 57 • Small startup • WireMock creator is a co-founder of the Inc. • WireMock Cloud - SaaS for end-to-end API mocking • Private beta: K8s Edition for managed / on-premises wiremock.io
58. Credits to • All WireMock contributors • WireMock Inc. Team • All Testcontainers contributors and AtomicJar folks • All FOSS contributors 58