Pass or Play: What does GenAI mean for the Java developer?

You’ve been tasked with implementing or learning AI, whether that includes LLMs, RAG, or MLOps. The sea of acronyms in this era of AI can feel like (and sometimes actually are) a black box for the complex logic and processes that underpin them. There’s so much to learn and not enough time.
In this session, we’ll explore how these technologies operate and the different practical ways GenAI is being used. We will explain common architectures for applications with GenAI, including RAG and GraphRAG. Live-code examples will show how Java developers can start using GenAI and help determine where they can be best applied. Come see how to play in the GenAI era.
Code: https://github.com/JMHReif/springai-goodreads

Transcript

1. Pass or Play What does GenAI mean for the Java developer? Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
2. Who is Jennifer Reif? Developer Advocate, Neo4j • Continuous learner • Conference speaker • Tech blogger • Other: geek Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
3. Photo by Matt Walsh on Unsplash AI Vector RAG LLM Algorithm Chaining Entity resolution Knowledge graph ML NLP GenAI Hallucination Embedding k-ANN Cosine similarity Euclidean distance Fine-tune Few-shot Grounding Model Prompt Semantic search Similarity Temperature Tokens Natural language ChatGPT Context window Agents GraphRAG MCP
4. Generative Arti fi cial Intelligence Artificial intelligence capable of generating text, images, or other data using generative models, often in response to prompts. Generative AI models learn the patterns and structure of their input training data and then generate new data that has similar characteristics. https://en.wikipedia.org/wiki/Generative_arti fi cial_intelligence
5. Large Language Model (LLM)
6. ChatGPT Public interface to LLM • Chat Generative Pre-trained Transformer • OpenAI, Nov 2022 • Natural language response • Predict next word • Feedback / reward to rank responses • Use cases: professional, personal, everything!
7. Are LLMs all that? Worth the hype? • Answers on probabilities • Trained on: • Public domain knowledge • Creative / arts • Human assistant • Task delegation Photo by Igor Omilaev on Unsplash
8. LLM issues • Lacking most recent data • Not always natural language • Language complexities, sarcasm, emotion • No sources • Hallucinations / Temperature • IP, bias, privacy
9. “Simply add an LLM” doesn’t work…
10. Strategies to improve LLM accuracy • Custom model • Fine-tuning / Few-shot learning • Retrieval Augmented Generation (RAG) • All of these involve training an LLM on speci fi c data!
11. RAG Pull data from external data sources • Retrieval • Data retrieved from database • Augmented • Augments response with facts • Generation • Response in natural language
12. Provide context …that we already have! • Plug and play with existing data • High-quality data • Internal value • Information in new ways
13. Benefits With RAG + LLM • Guide probability with context (grounding) • Focuses / narrows probability area • Adds to LLM knowledge • Reduce margin of error • How did the LLM get this answer? Photo by No Revisions on Unsplash
14. How to serve up what’s relevant And context window limit
15. Vectors in the physical realm https://www.mathsisfun.com/algebra/vectors.html
16. Vectors in the technical realm Kings and Queens king − man + woman ≈ queen king man wom an 1 king man wom an 2 queen? 3
17. Embeddings Convert data to a point in space • Series of numbers • 100s or 1000s of dimensions • Dimension = interesting feature / characteristic
18. LLMs take text, not databases Vectors
19. How do we search the vectors? Similarity search • Expensive queries (compare to every vector) • Approximate nearest neighbor (k-ANN) • Example: Library • Book classi fi cation - genre vs location of plot • Smaller search set = smaller retrieval time! Photo by Martin Adams on Unsplash
20. Provide prompt + context to LLM Vectors Vectors Vectors Vectors Similarity
21. Retrieval source options • Vector database • Relational (+ vectors) • NoSQL (+ vectors) • Graph (+ vectors) • Other: Directories / Websites / etc
22. GraphRAG Graph as data source in retrieval
23. What is a graph? Jennifer Michael ??
24. What is a graph? Company Jennifer Michael WORKED_FOR WORKED_FOR
25. What is a graph? Company Jennifer Michael School W ORKED_FOR WORKED_FOR ATTENDED ATTENDED
26. What is a graph? Degree Degree Company Jennifer Michael School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN
27. What is a graph? Adrian Degree Degree Company Jennifer Michael School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN ENROLLED_IN Degree C O M PLETED COM PLETED
28. What is a graph? Person Degree Degree Company Person Person School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN ENROLLED_IN Degree C O M PLETED COM PLETED Edward Jones Michael Jennifer SIUE Music CMIS CS Adrian
29. What is a graph? Answers through relationships • How many coworkers shared classes/degrees? • What are common degree journeys? • How many alumni re- enroll for higher degrees? • Who else went to a school and works for company? Person Degree Degree Company Person Person School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN ENROLLED_IN Degree C O M PLETED COM PLETED Edward Jones Michael Jennifer SIUE Music CMIS CS Adrian
30. Nodes (vertices) Objects or entities • Can have labels • May have properties Person Degree Degree Company Person Person School Degree Edward Jones Michael Jennifer SIUE Music CMIS CS Adrian
31. Relationships (edges) Connect entities • Must have type (label) • Must have direction • May have properties Person Degree Degree Company Person Person School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN ENROLLED_IN Degree C O M PLETED COM PLETED Edward Jones Michael Jennifer SIUE Music CMIS CS Adrian
32. Graphs add context and meaning Not just point-to-point, but HOW they connect
33. GraphRAG Graph as data source • Accuracy: • Extra context / related connections • Veri fi ability: • Check against understandable format • Explainability: • Trace path through graph for answer
34. Our demo Book recommendations • Search terms / topics • Book descrs -> User reviews • Lexical search -> semantic search
35. Provide prompt + context to LLM Vectors Vectors Vectors Vectors Similarity Graph
36. RAG architecture Prompt + Relevant Information LLM API LLM
 Chat API User Database Search Prompt Response Relevant Results / Documents 2 3 1 Database
37. Nothing is a silver bullet LLM is (of sorts) mind of its own • Can’t guarantee a consistent answer • Model decisions • Prompt construction • Context window limits
38. Let’s code! Vector search Data results Prompt phrase+data
41. Resources • Github repository (today’s code): github.com/JMHReif/springai-goodreads • GraphAcademy LLM courses: graphacademy.neo4j.com/categories/llms/ • Docs for Spring AI: docs.spring.io/spring-ai/reference/api/vectordbs/neo4j.html • Knowledge graph ebook: https://neo4j.com/whitepapers/developers-guide-how- to-build-knowledge-graph/ Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif