LangChain4j, Java, and You

LangChain4j, Java, & You Holly Cummins Sr. Technical Staff Member
IBM LJC Unconference 2025

@holly_cummins #Quarkus #IBM #RedHat 1849

@holly_cummins #Quarkus #IBM #RedHat 1859

@holly_cummins #Quarkus #IBM #RedHat data is the new oil 2006

@holly_cummins #Quarkus #IBM #RedHat 2006 data is the new oil

@holly_cummins #Quarkus #IBM #RedHat a data scientist. what do you
call a statistician who lives in san francisco?

@holly_cummins #Quarkus #IBM #RedHat

@holly_cummins #Quarkus #IBM #RedHat the large language model

@holly_cummins #Quarkus #IBM #RedHat deploy to prod rest api train
the model langchain use output in code standardize prompt structures switch from one LLM model to another memory (context)

@holly_cummins #Quarkus #IBM #RedHat but there’s still a problem

@holly_cummins #Quarkus #IBM #RedHat deploy to prod rest api train
the model use output in code standardize prompt structures switch from one LLM model to another memory (context) langchain langchain4j

@holly_cummins #Quarkus #IBM #RedHat this is an integration challenge

@holly_cummins #Quarkus #IBM #RedHat use output in code standardize prompt
structures switch from one LLM model to another memory (context) quarkus langchain4j langchain4j easy access to data dependency injection annotations live coding failover mockability observability

@holly_cummins #Quarkus #IBM #RedHat … but not live code more

@holly_cummins #Quarkus #IBM #RedHat bootstrapping <dependency> <groupId>io.quarkiverse.langchain4j</groupId> <artifactId>quarkus-langchain4j-openai</artifactId> </dependency> quarkus.langchain4j.openai.api-key=sk-...
configure an api key

@holly_cummins #Quarkus #IBM #RedHat @RegisterAiService interface Assistant { String chat(String
message); } define ai service use DI to instantiate assistant @Inject private final Assistant assistant;

@holly_cummins #Quarkus #IBM #RedHat @SystemMessage("You are a professional poet") @UserMessage("""
Write a poem about {topic}. The poem should be {lines} lines long. """) String writeAPoem(String topic, int lines); add context to the calls main message to send

@holly_cummins #Quarkus #IBM #RedHat interface TransactionExtractor { @UserMessage("Extract information about
a transaction from {it}") TransactionInfo extractTransaction(String text); } class TransactionInfo { @Description("full name") public String name; @Description("IBAN value") public String iban; @Description("Date of the transaction") public LocalDate transactionDate; @Description("Amount in dollars of the transaction") public double amount; } unmarshalling objects

@holly_cummins #Quarkus #IBM #RedHat memory @RegisterAiService(chatMemoryProviderSupplier = BeanChatMemoryProviderSupplier.class) interface AiServiceWithMemory
{ String chat(@UserMessage String msg); } --------------------------------- @Inject private AiServiceWithMemory ai; String userMessage1 = "Can you give a brief explanation of Kubernetes?"; String answer1 = ai.chat(userMessage1); String userMessage2 = "Can you give me a YAML example to deploy an app for this?"; String answer2 = ai.chat(userMessage2); possibility to customize memory provider remember previous interactions

@holly_cummins #Quarkus #IBM #RedHat @RegisterAiService(/*chatMemoryProviderSupplier = BeanChatMemoryProviderSupplier.class*/) interface AiServiceWithMemory {
String chat(@MemoryId Integer id, @UserMessage String msg); } --------------------------------- @Inject private AiServiceWithMemory ai; String answer1 = ai.chat(1,"I'm Frank"); String answer2 = ai.chat(2,"I'm Betty"); String answer3 = ai.chat(1,"Who Am I?"); default memory provider refers to conversation with id == 1, ie. Frank keep track of multiple parallel conversations

@holly_cummins #Quarkus #IBM #RedHat @RegisterAiService(tools = EmailService.class) public interface MyAiService
{ @SystemMessage("You are a professional poet") @UserMessage("Write a poem about {topic}. Then send this poem by email.") String writeAPoem(String topic); public class EmailService { @Inject Mailer mailer; @Tool("send the given content by email") public void sendAnEmail(String content) { mailer.send(Mail.withText("[email protected]", "A poem", content)); } } tools, functions, agents describe when to use the tool register the tool ties it back to the tool description

@holly_cummins #Quarkus #IBM #RedHat what could possibly go wrong? (why
raw models aren’t enough)

@holly_cummins #Quarkus #IBM #RedHat the demo fails you didn’t get
to see

@holly_cummins #Quarkus #IBM #RedHat Guardrail messages: A Noodle is not
a SNAKE.

solution: problem: @holly_cummins #Quarkus #IBM #RedHat hallucinations guard rails RAG

solution: problem: @holly_cummins #Quarkus #IBM #RedHat latency fault tolerance fallbacks

solution: problem: @holly_cummins #Quarkus #IBM #RedHat attacks enterprise security

solution: problem: @holly_cummins #Quarkus #IBM #RedHat cost observability smaller models
hybrid with rules engines

@holly_cummins #Quarkus #IBM #RedHat really: data is the new oil

@holly_cummins #Quarkus #IBM #RedHat - small models are cheaper +
greener - consider hybrid with rules engines - keep an eye on memory build-up; use no more context than needed - limit how much retrieved content is included in RAG queries

@holly_cummins #Quarkus #IBM #RedHat this is not magic … but
it is amazing

https://ibm.biz/BdnRNh Are you adding AI to existing Java applications? Are
you developing new AI applications in Java? We would love to talk to you! Use the QR code to provide your contact details and we’ll be in touch.

slides + demo repo @hollycummins.com http://hollycummins.com/langchain4j-ljc-unconference/ survey https://ibm.biz/BdnRNh

LangChain4j, Java, and You

LangChain4j, Java, and You

More Decks by Holly Cummins

Other Decks in Programming

Featured

Transcript