integrating existing models Do you really want to do • Transactions • Security • Scalability • Observability • … you name it in Python??? into enterprise- grade systems and applications Java??? 😯 … no seriously … why not Python? 🤔
A Large Language Model is at the core of any AI-Infused Application … but this is not enough. You also need: - Well crafted prompts guiding the LLM in the most precise and least ambiguous possible ways Prompts
A Large Language Model is at the core of any AI-Infused Application … but this is not enough. You also need: - Well crafted prompts guiding the LLM in the most precise and least ambiguous possible ways - A chat memory to "remember" previous interactions and make the AI service conversational Prompts Memory
A Large Language Model is at the core of any AI-Infused Application … but this is not enough. You also need: - Well crafted prompts guiding the LLM in the most precise and least ambiguous possible ways - A chat memory to "remember" previous interactions and make the AI service conversational - External tools (function calling) expanding LLM capabilities and take responsibility for deterministic tasks where generative AI falls short Prompts Memory Tools
A Large Language Model is at the core of any AI-Infused Application … but this is not enough. You also need: - Well crafted prompts guiding the LLM in the most precise and least ambiguous possible ways - A chat memory to "remember" previous interactions and make the AI service conversational - External tools (function calling) expanding LLM capabilities and take responsibility for deterministic tasks where generative AI falls short - Data/Knowledge sources to provide contextual information (RAG) and persist the LLM state Prompts Memory Tools Data Sources
Application A Large Language Model is at the core of any AI-Infused Application … but this is not enough. You also need: - Well crafted prompts guiding the LLM in the most precise and least ambiguous possible ways - A chat memory to "remember" previous interactions and make the AI service conversational - External tools (function calling) expanding LLM capabilities and take responsibility for deterministic tasks where generative AI falls short - Data/Knowledge sources to provide contextual information (RAG) and persist the LLM state - Guardrails to prevent malicious input and block wrong or unacceptable responses Prompts Memory Tools Data Sources
essence what makes an AI service also an Agent is the capability to collaborate with other Agents in order to perform more complex tasks and pursue a common goal
together is programmatically orchestrating them in fixed and predetermined workflows 4 basic patterns that can be used as building blocks to create more complex interactions - Sequence / Prompt chaining - Loop / Reflection - Parallelization - Conditional / Routing
a creative writer. Generate a draft of a story long no more than 3 sentence around the given topic. The topic is {topic}.""") @Agent("Generate a story based on the given topic") String generateStory(String topic); } public interface AudienceEditor { @UserMessage(""" You are a professional editor. Analyze and rewrite the following story to better align with the target audience of {audience}. The story is "{story}".""") @Agent("Edit a story to fit a given audience") String editStory(String story, String audience); } public interface StyleEditor { @UserMessage(""" You are a professional editor. Analyze and rewrite the following story to better fit and be more coherent with the {{style}} style. The story is "{story}".""") @Agent("Edit a story to better fit a given style") String editStory(String story, String style); Topic Story Audience Style Story Story
a creative writer. Generate a draft of a story long no more than 3 sentence around the given topic. The topic is {topic}.""") @Agent("Generate a story based on the given topic") String generateStory(String topic); } public interface AudienceEditor { @UserMessage(""" You are a professional editor. Analyze and rewrite the following story to better align with the target audience of {audience}. The story is "{story}".""") @Agent("Edit a story to fit a given audience") String editStory(String story, String audience); } public interface StyleEditor { @UserMessage(""" You are a professional editor. Analyze and rewrite the following story to better fit and be more coherent with the {{style}} style. The story is "{story}".""") @Agent("Edit a story to better fit a given style") String editStory(String story, String style); Topic, Audience, Style Story
a story based on the given topic, for a specific audience and in a specific style") String generateStory(String topic, String audience, String style); } Our Agent System Interface (API): var story = storyGenerator.generateStory( "dragons and wizards", "young adults", "fantasy");
.chatModel(myModel).outputKey( "story") .build(); var audienceEditor = agentBuilder(AudienceEditor. class) .chatModel(myModel).outputKey( "story").build(); var styleEditor = agentBuilder(StyleEditor. class) .chatModel(myModel).outputKey( "story").build(); var storyGenerator = AgenticServices.sequenceBuilder( StoryGenerator .class) .subAgents( creativeWriter , audienceEditor , styleEditor) .outputKey( "story").build(); Invoke the system using the StoryGenerator API
to communicate the results it produced read by another agent to retrieve the necessary to perform its task Records the sequence of invocations of all agents with their responses Provides agentic system wide context to an agent based on former agent executions Persistable via a pluggable SPI A collection of data shared among the agents participating in the same agentic system State topic audience style story
critical reviewer. Give a review score between 0.0 and 1.0 for the following story based on how well it aligns with the style '{style}'. Return only the score and nothing else. The story is: "{story}" """) @Agent("Score a story based on how well it aligns with a given style" ) double scoreStyle(String story, String style); }
mood); } public interface FoodExpert { @UserMessage(""" You are a great evening planner. Propose a list of 3 meals matching the given mood. The mood is {{mood}}. For each meal, just give the name of the meal. Provide a list with the 3 items and nothing else. """) @Agent List<String> findMeal(@V("mood") String mood); } public interface MovieExpert { @UserMessage(""" You are a great evening planner. Propose a list of 3 movies matching the given mood. The mood is {{mood}}. Provide a list with the 3 items and nothing else. """) @Agent List<String> findMovie(@V("mood") String mood); } EveningPlannerAgent eveningPlannerAgent = AgenticServices .parallelBuilder(EveningPlannerAgent.class) .subAgents(foodAgent, movieAgent) .outputKey("plans") .output(agenticScope -> { List<String> movies = agenticScope.readState("movies"); List<String> meals = agenticScope.readState("meals"); List<EveningPlan> moviesAndMeals = new ArrayList<>(); for (int i = 0; i < movies.size(); i++) { if (i >= meals.size()) { break; } moviesAndMeals.add(new EveningPlan(movies.get(i), meals.get(i))); } return moviesAndMeals; }); List<EveningPlan> plans = eveningPlannerAgent.plan("romantic");
} public enum RequestCategory { LEGAL, MEDICAL, TECHNICAL, UNKNOWN } public interface RouterAgent { @UserMessage(""" Analyze the user request and categorize it as 'legal', 'medical' or 'technical', The user request is: '{{request}}'. """) @Agent String askToExpert(@V("request") String request); } public interface MedicalExpert { @UserMessage(""" You are a medical expert. Analyze the user request under a medical point of view and provide the best possible answer. The user request is {{request}}. """) @Agent("A medical expert") String medical(@V("request") String request); } RouterAgent routerAgent = AgenticServices.agentBuilder(RouterAgent.class) .chatModel(myModel).outputKey("category").build(); MedicalExpert medicalExpert = AgenticServices .agentBuilder(MedicalExpert.class) .chatModel(myModel).outputKey("response").build()); LegalExpert legalExpert = ... TechnicalExpert techExpert = UntypedAgent expertsAgent = AgenticServices.conditionalBuilder() .subAgents(scope -> scope.readState("category",UNKNOWN) == MEDICAL, medicalExpert) .subAgents(scope -> scope.readState("category",UNKNOWN) == LEGAL, legalExpert) .subAgents(scope -> scope.readState("category",UNKNOWN) == TECHNICAL, techExpert) .build(); ExpertRouterAgent expertRouterAgent = AgenticServices .sequenceBuilder(ExpertRouterAgent.class) .subAgents(routerAgent, expertsAgent) .outputKey("response").build(); expertRouterAgent.ask("I broke my leg what should I do")
are stateless, meaning that they do not maintain any context or memory of previous interactions - AI Services can be provided with a ChatMemory, but this is local to the single agent, so in many cases not enough in a complex agentic system - In general an agent requires a broader context, carrying information about everything that happened in the agentic system before its invocation - That’s another task for the AgenticScope
are programmatically orchestrated through predefined code paths and workflows LLMs dynamically direct their own processes and tool usage, maintaining control over how they execute tasks Workflow Agents
All agentic systems explored so far orchestrated agents programmatically in a fully deterministic way - In many cases agentic system have to be more flexible and adaptive - An Autonomous Agentic AI system ◦ Takes autonomous decisions ◦ Decides iteratively which agent has to be invoked next ◦ Uses the result of previous interactions to determine if it is done and achieved its final goal ◦ Uses the context and state to generate the arguments to be passed to the selected agent
Response Supervisor Agent A Agent B Agent C Agent result + State Determine if done or next invocation Pool of agents Done Select and invoke (Agent Invocation)
result + State Determine if done or next invocation Pool of agents public record AgentInvocation( String agentName, Map<String, String> arguments) { } Done An Autonomous Agentic AI Case Study – Supervisor pattern
You are a planner expert that is provided with a set of agents. You know nothing about any domain, don't take any assumptions about the user request. Your role is to analyze the user request and decide which one of the provided agents to call next. You return an agent invocation consisting of the name of the agent and the arguments to pass to it. If no further agent requests are required, return an agentName of "done" and an argument named "response", where the value of the response argument is a recap of all the performed actions, written in the same language as the user request. Agents are provided with their name and description together with a list of applicable arguments in the format {name: description, [argument1, argument2]}. The comma separated list of available agents is: '{agents}'. Use the following optional supervisor context to better understand constraints, policies or preferences when creating the plan (can be empty): '{supervisorContext}'. """) @UserMessage("The user request is: '{req}'. The last received response is: '{lastResponse}'.") AgentInvocation plan(@MemoryId Object userId, String agents, String req, String lastResponse, String ctx); }
You are a planner expert that is provided with a set of agents. You know nothing about any domain, don't take any assumptions about the user request. Your role is to analyze the user request and decide which one of the provided agents to call next. You return an agent invocation consisting of the name of the agent and the arguments to pass to it. If no further agent requests are required, return an agentName of "done" and an argument named "response", where the value of the response argument is a recap of all the performed actions, written in the same language as the user request. Agents are provided with their name and description together with a list of applicable arguments in the format {name: description, [argument1, argument2]}. The comma separated list of available agents is: '{agents}'. Use the following optional supervisor context to better understand constraints, policies or preferences when creating the plan (can be empty): '{supervisorContext}'. """) @UserMessage("The user request is: '{req}'. The last received response is: '{lastResponse}'.") AgentInvocation plan(@MemoryId Object userId, String agents, String req, String lastResponse, String ctx); } Definition of “done”
You are a planner expert that is provided with a set of agents. You know nothing about any domain, don't take any assumptions about the user request. Your role is to analyze the user request and decide which one of the provided agents to call next. You return an agent invocation consisting of the name of the agent and the arguments to pass to it. If no further agent requests are required, return an agentName of "done" and an argument named "response", where the value of the response argument is a recap of all the performed actions, written in the same language as the user request. Agents are provided with their name and description together with a list of applicable arguments in the format {name: description, [argument1, argument2]}. The comma separated list of available agents is: '{agents}'. Use the following optional supervisor context to better understand constraints, policies or preferences when creating the plan (can be empty): '{supervisorContext}'. """) @UserMessage("The user request is: '{req}'. The last received response is: '{lastResponse}'.") AgentInvocation plan(@MemoryId Object userId, String agents, String req, String lastResponse, String ctx); } Passing the pool of agents
You are a planner expert that is provided with a set of agents. You know nothing about any domain, don't take any assumptions about the user request. Your role is to analyze the user request and decide which one of the provided agents to call next. You return an agent invocation consisting of the name of the agent and the arguments to pass to it. If no further agent requests are required, return an agentName of "done" and an argument named "response", where the value of the response argument is a recap of all the performed actions, written in the same language as the user request. Agents are provided with their name and description together with a list of applicable arguments in the format {name: description, [argument1, argument2]}. The comma separated list of available agents is: '{agents}'. Use the following optional supervisor context to better understand constraints, policies or preferences when creating the plan (can be empty): '{supervisorContext}'. """) @UserMessage("The user request is: '{req}'. The last received response is: '{lastResponse}'.") AgentInvocation plan(@MemoryId Object userId, String agents, String req, String lastResponse, String ctx); } User message of the planner
result Agentic Scope (Invocations +results) Pool of agents Done? Response Scorer Response Strategy State Scores Last, Score, Summary Input, response, action summary An Autonomous Agentic AI Case Study – Supervisor pattern
WithdrawAgent { @SystemMessage("You are a banker that can only withdraw US dollars (USD) from a user account.") @UserMessage("Withdraw {amountInUSD} USD from {withdrawUser}'s account and return the new balance.") @Agent("A banker that withdraw USD from an account") String withdraw(String withdrawUser, Double amountInUSD); } public interface CreditAgent { @SystemMessage("You are a banker that can only credit US dollars (USD) to a user account.") @UserMessage("Credit {amountInUSD} USD to {creditUser}'s account and return the new balance.") @Agent("A banker that credit USD to an account") String credit(String creditUser, Double amountInUSD); } public interface ExchangeAgent { @UserMessage(""" You are an operator exchanging money in different currencies. Use the tool to exchange {amount} {originalCurrency} into {targetCurrency} returning only the final amount provided by the tool as it is and nothing else. """) @Agent("A money exchanger that converts a given amount from the original to the target currency") Double exchange(String originalCurrency, Double amount, String targetCurrency); }
Mario's account to Kevin's account. Kevin's account has been credited with 115.0 USD, and the new balance is 1115.0 USD. The withdrawal of 115.0 USD from Mario's account has been completed, and the new balance is 885.0 USD. var result = bankSupervisor .invoke("Transfer 100 EUR from Mario's account to Kevin's"); System.out.println(result);
@SystemMessage( """ … """) @UserMessage("The user request is: '{req}'. The last received response is: '{lastResponse}'.") AgentInvocation plan(@MemoryId Object userId, String agents, String req, String lastResponse, String ctx); } AgentInvocation{agentName='exchange', arguments={originalCurrency=EUR, amount=100, targetCurrency=USD}} AgentInvocation{agentName='credit', arguments={creditUser=Kevin, amountInUSD=115.0}} AgentInvocation{agentName='withdraw', arguments={withdrawUser=Mario, amountInUSD=115.0}} AgentInvocation{agentName='done', arguments={response=100 EUR has been transferred from Mario's account to Kevin's account. Kevin's account has been credited with 115.0 USD, and the new balance is 1115.0 USD. The withdrawal of 115.0 USD from Mario's account has been completed, and the new balance is 885.0 USD.}
Pluggable Planner Workflow Supervisor GOAP P2P … Execution Layer Action Result State Agentic Scope Request Invoke Customizable by the framework (Quarkus) Agent A Agent B Agent C
initPlanningContext) { } default Action firstAction(PlanningContext planningContext) { return nextAction(planningContext); } Action nextAction(PlanningContext planningContext); } Any agentic pattern is simply a different specification of an execution plan for the subagents that it coordinates. The Planner interface generalizes this concept.
private List<AgentInstance> agents; private int agentCursor = 0; @Override public void init(InitPlanningContext initPlanningContext) { this.agents = initPlanningContext.subagents(); } @Override public Action nextAction(PlanningContext planningContext) { return agentCursor < agents.size() ? call(agents.get(agentCursor++)) : done(); } } Stores the subagents coordinated by this Planner Calls the next subagent in the sequence if any … … otherwise it’s done
Extractor prompt sign Horoscope Generator person horoscope sign Writer person writeup story Story Finder person story horoscope horoscope var horoscopeAgent = AgenticServices.plannerBuilder().outputKey("writeup") .subAgents(horoscopeGenerator, personExtractor, signExtractor, writer, storyFinder) .planner(GoalOrientedPlanner:: new) .build(); goal
prompt sign Horoscope Generator horoscope Writer person writeup Story Finder story horoscope person String writeup = horoscopeAgent.invoke( Map.of("prompt", "My name is Mario and my zodiac sign is pisces"));
Sign Extractor prompt sign Horoscope Generator horoscope Writer person Story Finder story writeup horoscope person String writeup = horoscopeAgent.invoke(Map.of( "prompt", "My name is Mario and my zodiac sign is pisces")); Style Scorer Style Editor Style Review Loop unedited writeup
Programmatic non-AI agents public class ExchangeOperator { @Agent("A money exchanger that converts a given amount of money from the original to the target currency" ) public Double exchange( @V("originalCurrency" ) String originalCurrency, @V("amount") Double amount, @V("targetCurrency" ) String targetCurrency) { // invoke the REST API to perform the currency exchange } }
.jpg){kind=small}
.jpg){kind=avatar}
.jpg){kind=avatar}
kdubois
kintotechdev
kasada
iwashi86
subroh0508
phaya72
shogogg
starfish719
soudai
oracle4engineer
shuta13
dynamis
nullnull
portentint
honzajavorek
skipperchong
reverentgeek
aleyda
3n
jnunemaker
cassininazir
szymonslowik
marketingsoph
rkendrick25
