$30 off During Our Annual Pro Sale. View Details »

A Hands-on AI Agent Guide with Gemma 💎, Ollama ...

Bilge Yücel
November 09, 2024

A Hands-on AI Agent Guide with Gemma 💎, Ollama 🦙 and more!

DevFest 2024 Istanbul

✨ Are you curious about building an AI agent from the ground up using open-source tools? ✨

Join this session, where we’ll create an intelligent AI agent with Gemma, Ollama, and other open source tools. We’ll start from scratch, exploring key components such as vector databases, RAG, prompt engineering, and evaluation to build a fully functional agent.

Whether you're new to AI development or looking to expand your toolkit, this session will provide practical insights, code examples, and everything you need to start building AI-driven applications.

Bilge Yücel

November 09, 2024
Tweet

More Decks by Bilge Yücel

Other Decks in Research

Transcript

  1. 🥑 Developer Relations Engineer at deepset 󰎲 🏗 Open source

    LLM Framework: Haystack 🎓 Sabanci University B.Sc. 🧠 KU Leuven M.Sc. 📍 Istanbul, Turkey ABOUT ME @BILGEYCL /IN/BILGE-YUCEL BILGEYUCEL Bilge Yucel
  2. AI AGENTS An agent in an LLM-powered system is an

    autonomous entity that uses a language model as its core to plan, reflect, store memory, and leverage external tools for executing complex tasks and interacting with external data sources efficiently
  3. AI AGENTS Overview of a LLM-powered autonomous agent system. Source:

    https://lilianweng.github.io/posts/2023-06-23-agent/
  4. • Can you mail me the latest Hacker News articles?

    ◦ Reason + Act = ReAct ◦ Tool calls, function calling ▪ find latest Hacker News articles ▪ generate an email ▪ send it AI AGENTS EXAMPLES
  5. • Generate JSON from {this} text ◦ Self-reflection: Generates a

    JSON, verifies it with the second iteration AI AGENTS EXAMPLES
  6. • Who is Taylor Swift? + When was she born?

    ◦ Memory: “When was Taylor Swift born?” AI AGENTS EXAMPLES
  7. GEMMA 2 • An open language model by Google •

    2b, 9b, 27b • 8192 tokens • wide range of tasks: question answering, commonsense reasoning, mathematics, science, and coding • Gemma License • More info in Gemma 2: Improving Open Language Models at a Practical Size
  8. OLLAMA OLLAMA • OSS to run models locally on macOS,

    Linux, and Windows • Models come with Ollama • Mostly 4-bit quantization • GPU -> CPU • Mostly for PoC
  9. OLLAMA How to use OLLAMA • Install Ollama to your

    computer • >> ollama pull <model_name> • >> ollama serve
  10. OLLAMA HAYSTACK • Haystack is an open source Python framework

    for building production-ready LLM applications • RAG, Semantic search, Agent… • Prototyping, evaluation, deployment, monitoring… • Building blocks: Components & Pipelines
  11. OLLAMA Web-Enhanced Self-Reflecting Agent • Retrieval Augmented Generation (RAG) •

    If not enough context, goes to web to enrich the context 🌐 • If enough context, finishes ✅
  12. OLLAMA PROMPT <start_of_turn>user {% if web_documents %} You were asked

    to answer the following query given the documents but the context was not enough. Here is the user question: {{ query }} Context: {% for document in documents %} {{document.content}} {% endfor %} {% for document in web_documents %} URL: {{document.meta.link}} TEXT: {{document.content}} — {% endfor %} Answer the question based on the given context. If you have enough context to answer this question, return your answer with the used links. If you don't have enough context to answer, say 'N0_ANSWER'. {% else %} Answer the following query based on the documents retrieved from Haystack's documentation. Documents: {% for document in documents %} {{document.content}} {% endfor %} Query: {{query}} If you have enough context to answer this question, just return your answer If you don't have enough context to answer, say 'N0_ANSWER'. {% endif %} <end_of_turn> <start_of_turn>model
  13. OLLAMA PROMPT <start_of_turn>user {% if web_documents %} You were asked

    to answer the following query given the documents but the context was not enough. Here is the user question: {{ query }} Context: {% for document in documents %} {{document.content}} {% endfor %} {% for document in web_documents %} URL: {{document.meta.link}} TEXT: {{document.content}} — {% endfor %} Answer the question based on the given context. If you have enough context to answer this question, return your answer with the used links. If you don't have enough context to answer, say 'N0_ANSWER'. {% else %} Answer the following query based on the documents retrieved from Haystack's documentation. Documents: {% for document in documents %} {{document.content}} {% endfor %} Query: {{query}} If you have enough context to answer this question, just return your answer If you don't have enough context to answer, say 'N0_ANSWER'. {% endif %} <end_of_turn> <start_of_turn>model