AI Agents, the new Frontier for LLMs

Know Large Language Models at your fingertips? Mastering Retrieval Augmented Generation to help an LLM search your documents? It's time to dive into the wonderful world of intelligent agents, the next frontier for LLMs!

In this session, we will first define what agents are, or at least what makes a system "agentic". We will explain the limitations of LLMs, key agent characteristics and patterns. Then, through concrete examples, we will implement various agents in Java, using the LangChain4j and ADK frameworks, to illustrate some typical agent patterns and to understand how to go beyond a simple LLM call to obtain responses that meet the needs of your users, or even to trigger actions with the surrounding system.

Are you ready for the next hype on agents? Come and discover it in this session!

Transcript

1. AI Agents -—-—-—-—- the new frontier for LLMs Guillaume Laforge Developer Advocate glaforge glaforge.dev @glaforge @glaforge.dev @glaforge@uwyn.net
2. What is an AI agent? Agent design patterns Use case #1: Agentic RAG Use case #2: Sci-Fi story agent Use case #3: Company profiler 01 02 03 04 05 📆 Agenda
3. LangChain4j
4. ADK — Agent Development Kit • Google’s framework to create AI agents • Model-agnostic (Gemini, Anthropic, Ollama, Lite-LLM, LangChain4j…) • Developer-centric • Simplified workflow (sequential, parallel…) • Supports a wide range of applications, from simple single-task agents to complex, multi-agent workflows
5. What is an AI Agent? 01 An LLM-powered System?
6. A general definition 🤖 AGENT 🧠 ENVIRONMENT sensors actuators perceive act Tools! An agent is a service that talks to an AI model to perform a goal-based operation using the tools and context it has.
7. The AI agent equation AI agent 🟰 LLM ➕ memory ➕ planning ➕ tool use
8. Key characteristics of AI agents THINK 🧠 Analyze user’s prompt & data, system prompt, to define a goal to reach 🗺 PLAN Check available tools, define the strategy to realize the requested goal REFLECT ♻ Evaluate & loop over the output, to fix errors, to suggest improvements 🎬 ACT RAG searches, API calls, code execution, invoke other agents, request human’s help
9. ➡ Autonomous The agent decides on its own ➡ Prompt-driven The plan is described explicitly & given in the prompt ➡ External workflow An external program or workflow drives the LLMs Who’s planning? Who’s planning? Me or you?
10. ➡ Autonomous The agent decides on its own Prompt-driven The plan is described explicitly & given in the prompt External workflow An external program or workflow drives the LLMs Who’s planning? AGENT 🗺 LLM
11. Autonomous The agent decides on its own ➡ Prompt-driven The plan is described explicitly & given in the prompt External workflow An external program or workflow drives the LLMs Who’s planning? AGENT 🗺 LLM 🏼 💼
12. Autonomous The agent decides on its own Prompt-driven The plan is described explicitly & given in the prompt ➡ External workflow An external program or workflow drives the LLMs Who’s planning? AGENT LLM
13. ➡ Autonomous The agent decides on its own ➡ Prompt-driven The plan is described explicitly & given in the prompt ➡ External workflow An external program or workflow drives the LLMs Who’s planning? Function call hallucinations, wrong & unordered steps Deterministic, explicit & predictable, easier to maintain
14. ➡ Autonomous The agent decides on its own ➡ Prompt-driven The plan is described explicitly & given in the prompt ➡ External workflow An external program or workflow drives the LLMs Who’s planning? More autonomy, higher agency, adapted for changing env. Stricter plan, requires code maintenance for evolution
15. Agent design patterns 02
16. Function calling Chatbot app Gemini What’s the weather like in Paris? It’s sunny in Paris! External API or service user prompt + getWeather(String) function contract call getWeather(“Paris”) for me please 󰚦 getWeather(“Paris”) {“forecast”:”sunny”} function response is {“forecast”:”sunny”} Answer: “It’s sunny in Paris!”
17. Control flows https://huyenchip.com/2025/01/07/agents.html SEQUENTIAL TASK A TASK B PARALLEL TASK A TASK B CONDITIONAL ROUTING TASK A TASK B TASK C LOOPING TASK A TASK B TASK C
18. Why? Trust, safety, compliance, accountability, clarification, uncertainty, dilemmas… HITL — Human In The Loop Decisions have to be made, Human! Important decisions & actions should be made by a human being! ⚠
19. ReAct pattern (Reason / Act) LLM-as-Judge Ask an LLM to check the output, suggest improvements, fix errors Reflection & self-critique Thought Observe Act
20. Agentic RAG 03
21. Embedding model calculate RAG Vector DB vector embeddings chunks DOCS split store vector + chunk ❶ INGESTION
22. LLM context + prompt + chunks Embedding model calculate RAG Chatbot app Vector DB vector embeddings chunks DOCS prompt vector embedding find similar answer ❶ INGESTION ❷ RETRIEVAL
23. Mintaka: A complex, natural, and multilingual dataset for end-to-end question answering. arXiv preprint arXiv:2210.01613 There are easy questions… and hard ones! Type Description Example Yes/No Answer is a Yes or No Has Lady Gaga ever made a song with Ariana Grande? Comparative Compare 2 items by an attribute Is Mont Blanc taller than Mount Rainier? Generic Simple questions Where was Michael Phelps born? Intersection Requires multiple conditions Which movie was directed by Denis Villeneuve and stars Timothee Chalamet? Ordinal Based on item's position in a list Who was the last Ptolemaic ruler of Egypt? Count Answer requires counting How many astronauts have been elected to Congress? Difference Contains a negation Which Mario Kart game did Yoshi not appear in? Superlative Max or Min of given attribute Who was the youngest tribute in the Hunger Games? Multi-hop Requires multiple steps to answer Who was the quarterback of the team that won Super Bowl 50?
24. Mintaka: A complex, natural, and multilingual dataset for end-to-end question answering. arXiv preprint arXiv:2210.01613 There are easy questions… and hard ones! Type Description Example Yes/No Answer is a Yes or No Has Lady Gaga ever made a song with Ariana Grande? Comparative Compare 2 items by an attribute Is Mont Blanc taller than Mount Rainier? Generic Simple questions Where was Michael Phelps born? Intersection Requires multiple conditions Which movie was directed by Denis Villeneuve and stars Timothee Chalamet? Ordinal Based on item's position in a list Who was the last Ptolemaic ruler of Egypt? Count Answer requires counting How many astronauts have been elected to Congress? Difference Contains a negation Which Mario Kart game did Yoshi not appear in? Superlative Max or Min of given attribute Who was the youngest tribute in the Hunger Games? Multi-hop Requires multiple steps to answer Who was the quarterback of the team that won Super Bowl 50?
25. Agentic RAG Berlin’s origins, population, geographic situation 🧠 Agentic Assistant ————————————— 1) Identify topics 2) Create questions 3) RAG search 4) Collect answers & generate final report 🛠 History/Geography Tool ———————————————— 1) Execute RAG search 2) Call topic assistant to summarize topic 🧠 Topic Assistant ———————————— 1) Study topic answers 2) Create a report summary on the topic TOPICAL REPORTS FINAL REPORT Vector database TOPICAL REPORT
26. Sci-Fi story authoring agent 04 This is my story! To infinity & beyond!
27. https://short-ai-story.web.app/
28. Agent workflow 🧠 Story writer ————————— Write a story with a title, and 5 chapters. Write a story about {{type}} 🦜 LangChain4j ☕ ——————————— Drives the workflow via code Firestore database Final story 🧠 Image prompter ———————————— Create an image prompt about: {{chapter}} 🧠 Imagen —————————— Generate an image about: {{imgPrompt}} 🧠 Image judge ———————— Pick best {{images}} for {{chapter}} For each image & chapter… 🧠 Text enhance ———————— Make chapter more legible {{chapter}} For each chapter… 🎲 Story about time travel, nanobots, aliens encounter, cyberpunk… PING! chapter 4 images imgPrompt
29. 05 ADK Agent Development Kit Hot off the press!
30. ADK — Google’s Agent Development Kit • New open source & code-first agent framework • Supports Gemini, and any LLM (via LiteLLM for Python & LangChain4j for Java) • Deployable anywhere ◦ your own server, cloud, K8S, etc. ◦ on Google Cloud Agent Engine (🐍) ◦ containerized on Google Cloud Run
31. ADK — Google’s Agent Development Kit • Multi-agent: a hierarchy of agents • Flexible orchestration: sequential, parallel, loop • Session state management: short & long term • MCP support for tool calling & A2A for multi-agent scenarios
32. ADK — Google’s Agent Development Kit • Integrations with ◦ LLM frameworks: LangChain(4j) & LlamaIndex ◦ Agent frameworks: LangGraph & CrewAI • Bi-directional multimodal streaming • Built-in ◦ Command Line Interface ◦ UI web-based console ◦ Evaluation capabilities (🐍)
33. Thanks for your attention (is all you need?) Guillaume Laforge Developer Advocate Ready for the AI agent future? glaforge glaforge.dev @glaforge @glaforge.dev @glaforge@uwyn.net
34. Illustrations courtesy of Imagen 3