Vector and GraphRAG: Accuracy and Explainability in GenAI Applications

Accuracy and explainability are critical in GenAI applications. When information from AI-integrated solutions is inaccurate, it can impact business, people’s health, financial decisions, and even legal policies, which causes cascading repercussions. Having the best data at the right time is vital.
LLMs are not able to handle this on their own, but retrieval augmented generation (RAG) can help by providing curated data as context to an LLM, guiding it to an appropriate answer. This session will explore how vector and graph RAG address the shortcomings of LLMs, explaining their shared functionality as well as some ways they handle it differently. Finally, we will see how to build a GenAI application with RAG to see these concepts in action.
Code: https://github.com/JMHReif/vector-graph-rag

Transcript

1. RAG: Accuracy and Explainability in GenAI Applications Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
2. Who is Jennifer Reif? Developer Advocate, Neo4j • Continuous learner • Technical speaker • Tech blogger, podcaster • Other: geek Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
3. Negative AI stories Even well-respected companies get it wrong • Hallucinating non-existent policy, legal cases • Chatbot produces Python • Legally binding vehicle o ff er • Harmful health advice • Threatening users • Inventing new language • Illegal activities (insider trading + local health laws)
4. Standalone LLM Doesn’t often work • Design: • human-consumable output • creative variation (probabilistic answers) • Problems: • too little detail, vague prompt • missing information (recent or private knowledge) • probabilistic ~= inconsistent
5. How do we avoid this? Add as much context as possible • Guide LLM to relevant ideas and content • Focuses / narrows search area • Adds to LLM knowledge • Reduces margin of error Photo by Ali Alauda on Unsplash
6. Demo - chatgpt.com
7. Who is Jennifer Reif? Public info IS available…
8. Let’s try Jennifer’s username Should be publicly fi ndable???
9. Adding some context… Pointing to a speci fi c source as guide
10. Giving it reference produces better results Now it knows where to look and what to look for
11. Layers of AI • More layers = better result • Complexity vs value
12. How to provide context …that we already have! • Can’t stu ff prompt with everything • Dynamic information • Plug and play with existing data • High-quality data
13. RAG architecture • Retrieval • Data retrieved from external source • Augmented • Augments response with facts • Generation • Response in natural language Prompt + Relevant Information LLM API LLM
 Chat API User Database Search Prompt Response Relevant Results / Documents 2 3 1 Database
14. Another layer Agentic systems • Multiple agents / tools • LLM decides which to use (and order) • Range: automation -> autonomous https://www.anthropic.com/engineering/building-e ff ective-agents
15. Retrieval source options • Vector database • Relational (+ vectors) • NoSQL (+ vectors) • Graph (+ vectors) • Other: Directories / Websites / etc
16. Our demo Book recommendations • Plug LLM into our curated book+review data set • Book descrs -> User reviews
17. Vector dbs
18. Embeddings / Vectors Convert data to a point in space • Series of numbers • 100s or 1000s of dimensions • Dimension = interesting feature / characteristic
19. How do we search the vectors? Similarity search • Expensive queries (compare to every vector) • Approximate nearest neighbor (k-ANN) • Proximity in vector space • Example: Library • Book classi fi cation - genre vs location of plot • Smaller search set = smaller retrieval time! Photo by Martin Adams on Unsplash
20. Pinecone: sample data Document{ id='10609bf6-b358-449e-a4d1-8a6f2a2f805d', text='As always a Page turner. may me think about where we are in our tech evolution.', media='null', metadata={ rating=4.0, book_id=18505765, distance=0.58059925 }, score=0.41940075159072876 }
21. Demo! https://www.pinecone.io/learn/vector-database/
22. Where do vectors fall flat? How do you… • Limited metadata / connections • Verify vector representations? • Explain how it got to answer? • Ensure highest relevance / accuracy?
23. Graph dbs
24. What is a graph? Jennifer Jacob ??
25. What is a graph? Company Jennifer Jacob WORKED_FOR WORKED_FOR
26. What is a graph? Company Jennifer Jacob School W ORKED_FOR WORKED_FOR ATTENDED ATTENDED
27. What is a graph? Degree Degree Company Jennifer Jacob School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN
28. What is a graph? Adrian Degree Degree Company Jennifer Jacob School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN ENROLLED_IN Degree C O M PLETED COM PLETED
29. What is a graph? Person Degree Degree Company Person Person School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN ENROLLED_IN Degree C O M PLETED COM PLETED Edward Jones Jacob Jennifer SIUE Music CMIS CS Adrian
30. What is a graph? Answers through relationships • How many coworkers shared classes/degrees? • What are common degree journeys? • How many alumni re- enroll for higher degrees? • Who else went to a school and works for company? Person Degree Degree Company Person Person School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN ENROLLED_IN Degree C O M PLETED COM PLETED Edward Jones Jacob Jennifer SIUE Music CMIS CS Adrian
31. Nodes (vertices) Objects or entities • Can have labels • May have properties Person Degree Degree Company Person Person School Degree Edward Jones Jacob Jennifer SIUE Music CMIS CS Adrian
32. Relationships (edges) Connect entities • Must have type (label) • Must have direction • May have properties Person Degree Degree Company Person Person School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN ENROLLED_IN Degree C O M PLETED COM PLETED Edward Jones Jacob Jennifer SIUE Music CMIS CS Adrian
33. Graphs add context and meaning Not just point-to-point, but HOW they connect
34. Neo4j: sample data Book[ id=18505765, title=Dark Matter (Star Carrier, #5), isbn=0062183990, isbn13=9780062183996, average_rating=3.93, authors=[ Author[name=Ian Douglas] ], reviewList=[ Review[ id=f78b825c122cc5b924957fafc7382bc1, text=As always a Page turner. may me think about where we are in our tech evolution., book_id=18505765, rating=4 ] ] ]
35. Vector vs Graph RAG:
36. How are they similar? • Store and index data for e ffi cient retrieval • Query data with queries • Semantic searches • Retrieve metadata / related data
37. Index differences Vector db vs Graph db • Top unit of vector data • Data upserted into index • Del index = del data • Search vs Semantic indexes • Use explicit proc in Cypher • Del index != del data https://glennas.wordpress.com/2011/03/13/understanding-graph-databases-marko-rodriguez/ https://docs.pinecone.io/guides/indexes/understanding-indexes
38. Storage practices Vector db vs Graph db • Prioritize stats • Some metadata / connections • Avoid large data types / values • Prioritize relationships • Value = metadata / connections • Avoid large types as properties Document{ id='10609bf6-b358-449e-a4d1-8a6f2a2f805d', text='As always a Page turner. may me think about where we are in our tech evolution.', media='null', metadata={ rating=4.0, book_id=18505765, distance=0.58059925 }, score=0.41940075159072876 } Book[ id=18505765, title=Dark Matter (Star Carrier, #5), isbn=0062183990, isbn13=9780062183996, average_rating=3.93, authors=[ Author[name=Ian Douglas] ], reviewList=[ Review[ id=f78b825c122cc5b924957fafc7382bc1, text=As always a Page turner. may me think about where we are in our tech evolution., book_id=18505765, rating=4 ] ] ]
39. Metadata / source information Structured + Unstructured Knowledge graph Unstructured Vectors Structured Semi-structured
40. Graphs = extra layer • Accuracy: • extra context / related info in connections • Veri fi ability: • check against understandable format • Explainability: • trace path through graph for answer
41. Our demo Book recommendations • Vector embeddings on Review text • Vector similarity search Reviews • Traverse graph from similar results • Less needle-in-a-haystack • More relevant!
42. Demo! https://thenewstack.io/graphrag-101-increasing-genai-accuracy-and-completeness/
45. Resources • Github repository (today’s code): github.com/JMHReif/vector-graph-rag • Docs for Spring AI: https://docs.spring.io/spring-ai/reference/api/vectordbs.html • GraphAcademy LLM courses: graphacademy.neo4j.com/categories/llms/ • Knowledge graph ebook: https://neo4j.com/whitepapers/developers-guide-how- to-build-knowledge-graph/ Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif