RAG: Accuracy and Explainability in GenAI Applications

Accuracy and explainability are critical in GenAI applications. When information from AI-integrated solutions is inaccurate, it can have severe and negative cascading repercussions. Having the best data at the right time is vital.
LLMs are not able to handle this on their own, but retrieval augmented generation (RAG) can help by providing curated data as context to an LLM, guiding it to an appropriate answer. This session will explore how vector and graph RAG address the shortcomings of LLMs, explaining their shared functionality as well as some ways they handle it differently. Finally, we will see how to build a GenAI application with RAG to see these concepts in action.
Code: https://github.com/JMHReif/vector-graph-rag

Transcript

1. RAG: Accuracy and Explainability in GenAI Applications Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
2. Who is Jennifer Reif? Developer Advocate, Neo4j • Continuous learner • Technical speaker • Tech blogger, podcaster • Other: geek Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif
3. Negative AI stories Even well-respected companies get it wrong • Hallucinating non-existent policy, legal cases • Chatbot produces Python • Legally binding vehicle o ff er • Harmful health advice • Threatening users • Inventing new language • Illegal activities (insider trading + local health laws)
4. Throw an LLM at it Doesn’t often work • LLM strengths: • producing human-consumable output • probabilistic answers • LLM weaknesses: • complex logic • recognizing nuances • evaluating/expressing sensitive topics
5. Demo - chatgpt.com
6. How do hallucinations happen? LLM limitations • Lacking recent data • Too broad / general search • Not enough context • Uncertainty
7. Adding context to the LLM What does it do? • Guides it • Focuses / narrows search area • Adds to LLM knowledge • Reduces margin of error
8. Demo - chatgpt.com
9. How to provide context …that we already have! • Plug and play with existing data • High-quality data • Internal value • Information in new ways
10. RAG architecture • Retrieval • Data retrieved from external source • Augmented • Augments response with facts • Generation • Response in natural language Prompt + Relevant Information LLM API LLM
 Chat API User Database Search Prompt Response Relevant Results / Documents 2 3 1 Database
11. Retrieval source options • Vector database • Relational (+ vectors) • NoSQL (+ vectors) • Graph (+ vectors) • Other: Directories / Websites / etc
12. Our demo Book recommendations • Search terms / topics • Book descrs -> User reviews • Lexical search -> semantic search • Less needle-in-a-haystack • More relevant!
13. Vector dbs
14. Embeddings / Vectors Convert data to a point in space • Series of numbers • 100s or 1000s of dimensions • Dimension = interesting feature / characteristic
15. Vectors in the physical realm https://www.mathsisfun.com/algebra/vectors.html
16. Vectors in the technical realm Kings and Queens king − man + woman ≈ queen king man wom an 1 king man wom an 2 queen? 3
17. How do we search the vectors? Similarity search • Expensive queries (compare to every vector) • Approximate nearest neighbor (k-ANN) • Example: Library • Book classi fi cation - genre vs location of plot • Smaller search set = smaller retrieval time! Photo by Martin Adams on Unsplash
18. Pinecone: sample data Document{ id='10609bf6-b358-449e-a4d1-8a6f2a2f805d', text='As always a Page turner. may me think about where we are in our tech evolution.', media='null', metadata={ rating=4.0, book_id=18505765, distance=0.58059925 }, score=0.41940075159072876 }
19. Where do vectors fall flat? How do you… • Verify vector representations? • Explain how it got to answer? • Ensure highest relevance / accuracy?
20. Graph dbs
21. What is a graph? Answers through relationships • Coworkers shared classes/degrees? • Common degree journeys? • Alumni re-enroll for higher degrees? • Who else went to my school? Person Degree Degree Company Person Person School ATTENDED ATTENDED W ORKED_FOR WORKED_FOR EN RO LLED_IN ENROLLED_IN ENROLLED_IN Degree C O M PLETED COM PLETED Edward Jones Jacob Jennifer SIUE Music CMIS CS Adrian
22. Graphs add context and meaning Not just point-to-point, but HOW they connect
23. What does it solve? Data problems • Documented path (not just data) • Answering how and why • Understanding/ fi nd hidden connections • Find alternates, impacts, etc.
24. Neo4j: sample data Book[ id=18505765, title=Dark Matter (Star Carrier, #5), isbn=0062183990, isbn13=9780062183996, average_rating=3.93, authors=[ Author[name=Ian Douglas] ], reviewList=[ Review[ id=f78b825c122cc5b924957fafc7382bc1, text=As always a Page turner. may me think about where we are in our tech evolution., book_id=18505765, rating=4 ] ] ]
25. Vector vs Graph RAG:
26. How are they similar? • Store and index data for e ffi cient retrieval • Query data with queries • Semantic searches • Retrieve metadata / related data
27. Index differences Vector db vs Graph db • Top unit of vector data • Data upserted into index • Del index = del data • Search vs Semantic indexes • Use explicit proc in Cypher • Del index != del data https://glennas.wordpress.com/2011/03/13/understanding-graph-databases-marko-rodriguez/ https://docs.pinecone.io/guides/indexes/understanding-indexes
28. Storage practices Vector db vs Graph db • Prioritize stats • Some metadata / connections • Avoid large data types / values • Prioritize relationships • Value = metadata / connections • Avoid large types as properties Document{ id='10609bf6-b358-449e-a4d1-8a6f2a2f805d', text='As always a Page turner. may me think about where we are in our tech evolution.', media='null', metadata={ rating=4.0, book_id=18505765, distance=0.58059925 }, score=0.41940075159072876 } Book[ id=18505765, title=Dark Matter (Star Carrier, #5), isbn=0062183990, isbn13=9780062183996, average_rating=3.93, authors=[ Author[name=Ian Douglas] ], reviewList=[ Review[ id=f78b825c122cc5b924957fafc7382bc1, text=As always a Page turner. may me think about where we are in our tech evolution., book_id=18505765, rating=4 ] ] ]
29. Graphs = extra layer • Accuracy: • extra context / related info in connections • Veri fi ability: • check against understandable format • Explainability: • trace path through graph for answer
30. Demo!
31. Resources • Github repository (today’s code): github.com/JMHReif/vector-graph-rag • Docs for Spring AI: https://docs.spring.io/spring-ai/reference/api/vectordbs.html • GraphAcademy LLM courses: graphacademy.neo4j.com/categories/llms/ • Knowledge graph ebook: https://neo4j.com/whitepapers/developers-guide-how- to-build-knowledge-graph/ Jennifer Reif jennifer.reif@neo4j.com @JMHReif github.com/JMHReif jmhreif.com linkedin.com/in/jmhreif