JavaZone 2025 - Non-deterministic? No problem! You can test it!

Testing is hard, which is why developers tend to avoid it. Testing non-deterministic things is even harder, which is unfortunate, since we're all writing AI-infused applications, and AI models are notoriously non-deterministic. What happens when the applications start using advanced features, such as RAG, tools, and agents? How do you test these applications? There must be some tools, technologies, and practices out there that can help, while not costing your organization lots of money!

Join Java Champions Oleg & Eric in this session as they explore some of these tools & technologies, such as Testcontainers, LangChain4j, Quarkus, and Ollama. They'll bring together Oleg's Testcontainers knowledge and Eric's testing obsessions, getting hands-on and show how you can incorporate these tools and technologies into your inner and outer loop processes.

You'll see how effortlessly Quarkus integrates with Testcontainers, and how Testcontainers can be used in conjunction with popular LLMs when writing tests. You'll also learn about how to use containers to extend your testing into your CI environments, so you can always be sure that if your tests are green you're good to go!

Eric Deandrea
Eric Deandrea is a Java Champion & Senior Principal Developer Advocate at IBM, focusing on application development technologies. Eric has over 25 years of experience designing and building Java-based solutions and developer training programs. He is a contributor to various OSS projects, including Quarkus, Spring, LangChain4j, WireMock, and Microcks, as well as a speaker at many public events and user groups around the world. Eric recently put his Quarkus and Spring knowledge to use by publishing his first book, 'Quarkus for Spring Developers (https://red.ht/quarkus-spring-devs).' Outside of work, Eric enjoys boating on the lakes of New Hampshire, ice hockey, and martial arts, in which he holds a black belt in Kempo Karate.

Oleg Šelajev
Oleg Šelajev is a developer advocate at Docker working mainly on developer productivity, Testcontainers, improving how we set up local development environments and tests, and building applications with AI parts. Developer. Author. Speaker. Java Champion. Docker captain. Loves all languages.

Transcript

1. @shelajev @edeandrea Eric Deandrea, Java Champion Oleg Šelajev, Java Champion Non-deterministic? No problem! You can test it!
2. @shelajev @edeandrea • Java Champion • 26+ years software development experience • Contributor to Open Source projects Quarkus Spring Boot, Spring Framework, Spring Security LangChain4j (& Quarkus LangChain4j) Wiremock Microcks • Boston Java Users ACM Chapter Board Member & Vice Chair • Published Author • Cat lover • Black belt in martial arts About Us
3. @shelajev @edeandrea • Showcase & explain Quarkus, how it enables modern Java development & the Kubernetes- native experience • Introduce familiar Spring concepts, constructs, & conventions and how they map to Quarkus • Equivalent code examples between Quarkus and Spring as well as emphasis on testing patterns & practices 3 https://red.ht/quarkus-spring-devs
4. @shelajev @edeandrea • Surprisingly, also, a Java Champion • 18+ years software development experience • ~11 years Developer Advocate • Loves to stare at the code of Open Source projects Quarkus Spring Boot LangChain4j Microcks Testcontainers (sometimes contributes bugs too!) • Half decent chess player • Allergic to cats About Us
5. @shelajev @edeandrea What are you hoping to learn here? What are you hoping to learn here? What are you going to leave with?
6. @shelajev @edeandrea @shelajev @edeandrea Did we get better or worse with this release?
7. @shelajev @edeandrea
8. @shelajev @edeandrea What’s changed in the last 6 months? • Standardization ◦ Or lack thereof (lots of competing standards)? • Distributed • Orchestrated • Smaller models • Agentic
9. @shelajev @edeandrea @shelajev @edeandrea “Types” of AI
10. @shelajev @edeandrea AI replacing humans
11. @shelajev @edeandrea AI replacing software
12. @shelajev @edeandrea
13. @shelajev @edeandrea How does your DevOps evolve when you infuse your applications with AI?
14. @shelajev @edeandrea
15. @shelajev @edeandrea DevOps Evolution Dev Ops Evaluate Data ML
16. @shelajev @edeandrea https://github.com/edeandrea/non-deterministic-no-problem
17. @shelajev @edeandrea Chat Bot Web Socket Claim AI Assistant Claim Status Notification Generate Email AI Assistant Output Guardrails Politeness AI Assistant AI replacing humans AI replacing software https://github.com/edeandrea/non-deterministic-no-problem Code I write Voodoo magic Legend Input Guardrails
18. @shelajev @edeandrea
19. @shelajev @edeandrea @shelajev @edeandrea Vanilla AI
20. @shelajev @edeandrea Application Database Application Service CRUD application Microservice Application Model AI-Infused application What’s the difference between these?
21. @shelajev @edeandrea Application Database Application Service CRUD application Microservice Application Model AI-Infused application Integration Points What’s the difference between these?
22. @shelajev @edeandrea Testing AI Replacing Humans Playwright chat bot Recording done by Playwright while testing a chat bot
23. @shelajev @edeandrea Testing AI Replacing Humans 28
24. @shelajev @edeandrea Rethink your approach
25. @shelajev @edeandrea Signal from tests: - stuff needs fixing - confident to release
26. @shelajev @edeandrea Signal from tests: - stuff needs fixing - confident to release Purpose of tests: - prevent breaking prod - continuously improve your app
27. @shelajev @edeandrea https://www.upworthy.com/prankster-tricks-a-gm-dealership-chatbot-to-sell-him-a-76000-chevy-tahoe-for-1-rp3 https://www.cbsnews.com/news/aircanada-chatbot-discount-customer https://www.bbc.com/news/technology-35902104 https://www.spiceworks.com/tech/artificial-intelligence/news/meta-blender-bot-3-controversy https://www.linkedin.com/posts/stephanjanssen_princoming-activity-7285987635628507136-9Ubw
28. @shelajev @edeandrea What does failure look like? What do we need to do differently?
29. @shelajev @edeandrea Application Database Application Service CRUD application Microservice Application Model AI-Infused application Integration Points Observability (metrics, tracing, logs, auditing) Fault Tolerance (timeout, bulkhead, circuit breaker, rate limiting, fallbacks, …) What’s the difference between these?
30. @shelajev @edeandrea https://library.wiremock.org/catalog/api/o/openai.com/openai-com https://mockgpt.wiremock.io https://docs.quarkiverse.io/quarkus-wiremock/dev
31. @shelajev @edeandrea What happens when we do this?
32. @shelajev @edeandrea What happens when we do this?
33. @shelajev @edeandrea
34. @shelajev @edeandrea
35. @shelajev @edeandrea Stupidity will always find a way…
36. @shelajev @edeandrea Stupidity Prompt: Please return a JSON document in the following format: { “name: “String”, “countryOfOrigin”: “String”} Response: Sure I’d love to give you some JSON! Here it is: ```json { “name”: “Eric”, “countryOfOrigin”: “USA” } ```
37. @shelajev @edeandrea Prompt Engineering and team topologies I said JSON!
38. @shelajev @edeandrea Guardrails - Out of the box in LangChain4j & Quarkus! - Functions used to validate the input and output of the model - Detect invalid input or output - Detect prompt injection - Detect hallucination - Chain of guardrails - Sequential - Stop at first failure
39. @shelajev @edeandrea Retry and Reprompt Output guardrails can have 4 different outcomes: - Success - Response is passed to the caller or next guardrail - Fatal - Stop and throw an exception - Retry - Call the model again with the same context we never know ;-) - Reprompt - Call the model again with another message in the model indicating how to fix the response
40. @shelajev @edeandrea
41. @shelajev @edeandrea Stupidity will always find a way…
42. @shelajev @edeandrea Observability
43. @shelajev @edeandrea Observability Collect metrics - Exposed as Prometheus - Track token usage & cost OpenTelemetry Tracing - Trace interactions with the LLM Auditing - Track of interactions with the LLM - Ability to replay & re-score interactions
44. @shelajev @edeandrea @shelajev @edeandrea Putting it all together
45. @shelajev @edeandrea • Like static analysis ◦ Are we getting better or worse over time? • Remember observability? Systematic Eval: are you getting better or worse? https://docs.quarkiverse.io/quarkus-langchain4j/dev/testing.html
46. @shelajev @edeandrea Selection decisions are not application based https://artificialanalysis.ai/models
47. @shelajev @edeandrea @shelajev @edeandrea Takeaways
48. @shelajev @edeandrea • LangChain4j & Quarkus are awesome! Get simple problems out of the way first • Naming things is still the hardest thing in computer science • Don’t forget your craft: DevOps process is there to help • Write tests, expect change and failure, deploy often • AI is just an API call Actual takeaways
49. @shelajev @edeandrea https://quarkus.io @quarkusio https://quarkusio.zulipchat.com @quarkus.io
50. @shelajev @edeandrea @shelajev @edeandrea Thank You! Slides https://bit.ly/jz25-non-deterministic