Mastering Generative AI: From Word Embeddings to Custom AI Agents

Turkiye Java Community - DefineX Meetup - 25.05.2024

https://kommunity.com/turkiye-java-community/events/tjc-definex-developer-meetups-af0916f1

Abstract
What are embeddings? How can an LLM answer questions about your data? How challenging is it to implement your own AI agent?

In this talk, we’ll start from scratch, exploring key NLP concepts such as vector databases and word embeddings, while gaining a deeper understanding of large language models and the art of prompting. We’ll then implement our first RAG application together and explore techniques to enhance it. Building on this foundation, we’ll advance our GenAI knowledge to learn how to implement an agent capable of connecting to any tool we choose.

By the end of this presentation, you’ll gain a solid grasp of generative AI fundamentals and have a clear understanding of how to implement an LLM-based system.

Transcript

1. Bilge Yücel TJC & DefineX Mastering Generative AI From Word Embeddings to Custom AI Agents
2. 01 - What is it? • 🥑 Developer Relations Engineer at deepset • 🏗 Open source LLM Framework: Haystack • 📍 Istanbul, Turkey Bilge Yücel Twitter: @bilgeycl Linkedin: Bilge Yucel GitHub: @bilgeyucel Hi! 👋
3. NLP?
4. NLP?
5. NLP?
6. NLP - Natural Language Processing
7. Agenda 01 - Text Embeddings 02 - Vector Databases 03 - Retrieval 04 - LLMs 05 - RAG 06 - Agents & Function Calling
8. 01 Text Embeddings
9. Text Embeddings/Text Vectors to be or not to be • Manageable by computers • Different techniques: ◦ Sparse: TF-IDF, BM25... ◦ Dense: Trained models (Sentence Transformers, Cohere, OpenAI...) • Come in different dimensions
10. Text Embeddings - 2 dimensional
11. Text Embeddings - Semantic Similarity
12. 02 Vector Databases
13. • Databases that store high-dimensional vectors • Optimized for vectors: ◦ Vector search ◦ CRUD operations ◦ Metadata filtering Vector Databases
14. Databases https://docs.haystack.deepset.ai/docs/choosing-a-document-store
15. 03 Retrieval
16. Retrieval Query • Getting the most relevant information to the query • Used for semantic search, question answering and more
17. 04 LLMs
18. Large Language Models (LLMs)
19. Large Language Models (LLMs) • Big language models • More data & more parameter • Prompt → Human-like output • Text generation: summarization, generative QA, writing code, chat…
20. Model Providers - Which one?
21. LLMs being LLMs • Not conscious • Use carefully https://x.com/hrrcnes/status/1793335665117044990
22. LLMs know everything?
23. LLM: Limitations • LLMs do not know the answer to everything • But they are good at following instructions • We can help them in their task by giving them the relevant context + instruction
24. 05 Retrieval Augmented Generation
25. Retrieval Augmented Generation
26. Retrieval Augmented Generation
27. • Haystack is an open source Python framework for building production-ready LLM applications • Prototyping, evaluation, deployment, monitoring… • Building blocks: Components & Pipelines Haystack
28. Haystack
29. Indexing Pipeline
30. Indexing Pipeline
31. Indexing Pipeline
32. Indexing Pipeline
33. RAG Pipeline
34. RAG Pipeline
35. RAG Pipeline
36. RAG Pipeline
37. RAG Pipeline
38. RAG Pipeline
40. 06 Agents
41. Agents • Complex systems (multiple LLMs) • Prompting: Chain of Thought/ReAct • Generate accurate responses to complex queries • Might use tools for answers • Use cases: chat bot, personal assistants
42. Agents - CoT Chain-of-Thought Prompting Elicits Reasoning in Large Language Models - https://arxiv.org/pdf/2201.11903
43. Agents - CoT Chain-of-Thought Prompting Elicits Reasoning in Large Language Models - https://arxiv.org/pdf/2201.11903
44. REACT: SYNERGIZING REASONING AND ACTING IN LANGUAGE MODELS - https://arxiv.org/pdf/2210.03629 Agents - CoT
45. Agents - Reason + Act (ReAct) REACT: SYNERGIZING REASONING AND ACTING IN LANGUAGE MODELS - https://arxiv.org/pdf/2210.03629
46. Agents - ReAct Prompt
47. 06.5 Function Calling
48. Function Calling • Passing prompt + functions/tools (name, description, parameters) • User query -> function name + JSON object for arguments • Use arguments to call the function • Voila! 🎉
49. Function Calling - Tools https://arxiv.org/pdf/2205.00445
50. Function Calling - Tool definition
51. Function Calling
52. Agents Search Google and answer: why did Elon Musk sue OpenAI?
53. Agents Search Google and answer: why did Elon Musk sue OpenAI?
54. Agents Search Google and answer: why did Elon Musk sue OpenAI?
55. Agents Search Google and answer: why did Elon Musk sue OpenAI? Elon Musk sued OpenAI for breaching its founding agreement and diverging from its original nonprofit mission….
56. Agents Using TinyURL create a shortened URL from this long URL https://haystack.deep set.ai/blog/haystack- 2-release https://tinyurl.com/hjr6yttr
57. Agents
58. Recap 01 - Text Embeddings 02 - Vector Databases 03 - Retrieval 04 - LLMs 05 - RAG 06 - Agents & Function Calling
59. @bilgeycl Bilge Yücel bilgeyucel Thank you! haystack.deepset.ai