JChampionsConf 2026 - Non-deterministic? No problem! You can test it!

Testing is hard, which is why developers tend to avoid it. Testing non-deterministic things is even harder, which is unfortunate, since we're all writing AI-infused applications, and AI models are notoriously non-deterministic. What happens when the applications start using advanced features, such as RAG, tools, and agents? How do you test these applications? There must be some tools, technologies, and practices out there that can help, while not costing your organization lots of money!

Join Java Champions Oleg & Eric in this session as they explore some of these tools & technologies, such as Testcontainers, LangChain4j, Quarkus, and Ollama. They’ll bring together Oleg’s Testcontainers knowledge and Eric’s testing obsessions, getting hands-on and show how you can incorporate these tools and technologies into your inner and outer loop processes.

You’ll see how effortlessly Quarkus integrates with Testcontainers, and how Testcontainers can be used in conjunction with popular LLMs when writing tests. You’ll also learn about how to use containers to extend your testing into your CI environments, so you can always be sure that if your tests are green you’re good to go!

Transcript

1. @shelajev @edeandrea Eric Deandrea, Java Champion Oleg Šelajev, Java Champion Non-deterministic? No problem! You can test it!
2. @shelajev @edeandrea • Java Champion • 26+ years software development experience • Contributor to Open Source projects Quarkus Spring Boot, Spring Framework, Spring Security LangChain4j (& Quarkus LangChain4j) Wiremock Microcks • Boston Java Users ACM Chapter Board Member & Vice Chair • Published Author • Cat lover • Black belt in martial arts About Us
3. @shelajev @edeandrea • Showcase & explain Quarkus, how it enables modern Java development & the Kubernetes-native experience • Introduce familiar Spring concepts, constructs, & conventions and how they map to Quarkus • Equivalent code examples between Quarkus and Spring as well as emphasis on testing patterns & practices 3 https://red.ht/quarkus-spring-devs
4. @shelajev @edeandrea • also a Java Champion • Java developer • ~11 years Developer Advocate • Loves to stare at Open Source projects • Allergic to cats About Us
5. @shelajev @edeandrea What are you hoping to learn here? What are you hoping to learn here? What are you going to leave with?
6. @shelajev @edeandrea @shelajev @edeandrea Did we get better or worse with this release?
7. @shelajev @edeandrea
8. @shelajev @edeandrea What’s changed in the last year? • Standardization ◦ Or lack thereof (lots of competing standards)? • Distributed • Orchestrated • Agentic Agents • Agents • Agentic Agents • Autonomous Agents Smells like microservices?
9. @shelajev @edeandrea @shelajev @edeandrea “Types” of AI
10. @shelajev @edeandrea AI replacing humans
11. @shelajev @edeandrea AI replacing software
12. @shelajev @edeandrea
13. @shelajev @edeandrea How does your DevOps evolve when you infuse your applications with AI?
14. @shelajev @edeandrea
15. @shelajev @edeandrea DevOps Evolution Dev Ops Release Deploy Operate Monitor Plan Code Build Test Train Evaluate Deploy Collect Evaluate Curate Analyze Data ML
16. @shelajev @edeandrea https://github.com/edeandrea/non-deterministic-no-problem
17. @shelajev @edeandrea Chat Bot Web Socket Claim AI Assistant Claim Status Notification Tool invocation Generate Email AI Assistant Output Guardrails Politeness AI Assistant AI replacing humans AI replacing software https://github.com/edeandrea/non-deterministic-no-problem Code I write Voodoo magic Legend RAG Retrieval Input Guardrails Could be an agent? Could be an agent?
18. @shelajev @edeandrea AI: String -> String Agent: String -> IO String
19. @shelajev @edeandrea
20. @shelajev @edeandrea @shelajev @edeandrea Vanilla AI
21. @shelajev @edeandrea Application Database Application Service CRUD application Microservice Application Model AI-Infused application What’s the difference between these?
22. @shelajev @edeandrea Application Database Application Service CRUD application Microservice Application Model AI-Infused application Integration Points What’s the difference between these?
23. @shelajev @edeandrea Testing AI Replacing Humans
24. @shelajev @edeandrea Testing AI Replacing Humans 24
25. @shelajev @edeandrea Rethink your approach
26. @shelajev @edeandrea Signal from tests: ❌ - stuff needs fixing ✅ - confident to release
27. @shelajev @edeandrea Signal from tests: - stuff needs fixing - confident to release Purpose of tests: ❌ - prevent breaking prod ✅ - continuously improve your app
28. @shelajev @edeandrea https://www.upworthy.com/prankster-tricks-a-gm-dealership-chatbot-to-sell-him-a-76000-chevy-tahoe-for-1-rp3 https://www.cbsnews.com/news/aircanada-chatbot-discount-customer https://www.bbc.com/news/technology-35902104 https://www.spiceworks.com/tech/artificial-intelligence/news/meta-blender-bot-3-controversy https://www.linkedin.com/posts/stephanjanssen_princoming-activity-7285987635628507136-9Ubw
29. @shelajev @edeandrea Testing Strategies
30. @shelajev @edeandrea What does failure look like? What do we need to do differently?
31. @shelajev @edeandrea This isn’t the answer!
32. @shelajev @edeandrea Application Database Application Service CRUD application Microservice Application Model AI-Infused application Integration Points Observability (metrics, tracing, logs, auditing) Fault Tolerance (timeout, bulkhead, circuit breaker, rate limiting, fallbacks, …) What’s the difference between these?
33. @shelajev @edeandrea https://library.wiremock.org/catalog/api/o/openai.com/openai-com https://mockgpt.wiremock.io https://docs.quarkiverse.io/quarkus-wiremock/dev
34. @shelajev @edeandrea https://www.trtworld.com/europe/swedish-recycling-so-successful-it-is-importing-rubbish-24491
35. @shelajev @edeandrea What happens when we do this?
36. @shelajev @edeandrea What happens when we do this? Is this an error?
37. @shelajev @edeandrea
38. @shelajev @edeandrea
39. @shelajev @edeandrea Stupidity will always find a way…
40. @shelajev @edeandrea Stupidity Prompt: Please return a JSON document in the following format: { “name: “String”, “countryOfOrigin”: “String”} Response: Sure I’d love to give you some JSON! Here it is: ```json { “name”: “Eric”, “countryOfOrigin”: “USA” } ```
41. @shelajev @edeandrea Prompt Engineering and team topologies I said JSON!
42. @shelajev @edeandrea Guardrails - Out of the box in LangChain4j & Quarkus! - Functions used to validate the input and output of the model - Detect invalid input or output - Detect prompt injection - Detect hallucination - Chain of guardrails - Sequential - Stop at first failure
43. @shelajev @edeandrea Retry and Reprompt Output guardrails can have 4 different outcomes: - Success - Response is passed to the caller or next guardrail - Fatal - Stop and throw an exception - Retry - Call the model again with the same context we never know ;-) - Reprompt - Call the model again with another message in the model indicating how to fix the response
44. @shelajev @edeandrea
45. @shelajev @edeandrea Stupidity will always find a way…
46. @shelajev @edeandrea Observability
47. @shelajev @edeandrea Observability Collect metrics - Exposed as Prometheus - Track token usage & cost OpenTelemetry Tracing - Trace interactions with the LLM Auditing - Track of interactions with the LLM - Ability to replay & re-score interactions
48. @shelajev @edeandrea @shelajev @edeandrea Putting it all together
49. @shelajev @edeandrea • Like static analysis ◦ Are we getting better or worse over time? • Remember observability? Systematic Eval: are you getting better or worse? https://docs.quarkiverse.io/quarkus-langchain4j/dev/testing.html
50. @shelajev @edeandrea Selection decisions are not application based https://artificialanalysis.ai/models
51. @shelajev @edeandrea @shelajev @edeandrea Takeaways
52. @shelajev @edeandrea • LangChain4j & Quarkus are awesome! Get simple problems out of the way first • Naming things is still the hardest thing in computer science • Don’t forget your craft: DevOps process is there to help • Write tests, expect change and failure, deploy often • AI is just an API call Actual takeaways
53. @shelajev @edeandrea https://quarkus.io @quarkusio https://quarkusio.zulipchat.com @quarkus.io
54. @shelajev @edeandrea The Docker Model Runner Run models next to your other containerized services using the tools you're already using models: gemma3: model: ai/gemma3:4B-F16 services: app: models: gemma3: endpoint_var: OPENAI_BASE_URL model_var: OPENAI_MODEL compose.yaml
55. @shelajev @edeandrea The MCP Catalog Run MCP servers using containers without worrying about runtimes or installs anymore
56. @shelajev @edeandrea @shelajev @edeandrea Thank You! Slides https://bit.ly/jz25-non-deterministic