Share
Embed
RAGHack: Kickoff and RAG 101
An intro to RAGHack, a global hackathon to develop apps using LLMs and RAG. A large language model (LLM) like GPT-4 can be used for summarization, translation, entity extraction, and question-answering. Retrieval Augmented Generation (RAG) is an approach that sends context to the LLM so that it can provide grounded answers. RAG apps can be developed on Azure using a wide range of programming languages and retrievers (such as AI Search, Cosmos DB, PostgreSQL, and Azure SQL). Get an overview of RAG in this session before diving deep in our follow-up streams.
Pamela Fox
September 03, 2024
Transcript
43 slides
-
RAGHack streams Week 1: September 3 - 8 Week 2:
September 9 - 14 3 4 5 6 7 8 TUESDAY WEDNESDAY THURSDAY FRIDAY SATURDAY SUNDAY RAG 101 .NET Azure AI Studio Python Langchain4J LangchainJS Responsible AI MongoDB Azure AI Search PostgreSQL Azure SQL GraphRAG Multi-channels Hack! Hack! Hack! Hack! Hack! Hack! 10 11 12 13 14 15 TUESDAY WEDNESDAY THURSDAY FRIDAY SATURDAY SUNDAY VSCode Agent Agentic RAG Code Interpreter AI Studio Advanced AutoGen Data Access Control Fine Tuning Model catalog Evaluations Hack! Hack! Hack! Hack! Hack! Hack! 9 MONDAY Semantic Kernel Spring AI Vision models Internationalization Hack! Hack! Hack! Submissions due September 16, 11:59 PM PT! 9 MONDAY aka.ms/RAGhack -
RAGHack streams in more languages! Spanish: aka.ms/RagHack2024Espanol Portuguese: aka.ms/S-1374 Chinese:
aka.ms/S-1380 RAG: Generación Aumentada de Recuperación Prácticas recomendadas de Azure AI Search AI Multi-Agentes: Patrones, Problemas y Soluciones RAG (Geração Aumentada de Busca) no Azure Construindo RAG com Azure AI Studio e Python Implantando RAG com .NET e Azure Developer CLI Global RAG Hack Together Create RAG apps with Azure AI SDK Create RAG applications with AI Toolkit VSCode Extension Intro to GraphRAG -
RAGHack prizes •Best overall •Best in JavaScript/TypeScript •Best in Java
•Best in .NET •Best in Python •Best use of AI Studio •Best use of AI Search •Best use of PostgreSQL •Best use of Cosmos DB •Best use of Azure SQL Each winning team will receive a cash prize of $500 -
RAGHack submission process 1. Register by registering for any of
the RAGHack live streams linked from aka.ms/raghack/streams 2. Before September 16, 11:59 PM PT, submit your hack by filing an issue in aka.ms/raghack and fill out all the fields. A project repo and demo video is required, a deployed endpoint is optional. Include team members if you have any. 3. Wait a few weeks for judges to score! -
RAG: Retrieval Augmented Generation 101 Pamela Fox, Python Cloud Advocate
@pamelafox pamelafox.org -
LLM: Large Language Model An LLM is a model that
is so large that it achieves general-purpose language understanding and generation. Review: This movie sucks. Sentiment: negative Review: I love this movie: Sentiment: Input LLM positive Output -
Using LLMs on Azure Creator Models How to use on
Azure? OpenAI GPT3.5, GPT4, GPT4o, ada-002, text-embedding-3 Azure OpenAI, GitHub Models Microsoft Phi-3, Phi-3mini Azure AI Model Catalog, GitHub Models Mistral Nemo, Small, Large Cohere Command R, Command R+, Embed Meta LlaMA 3.1-8B, 70B, 405B https://github.com/marketplace/models https://azure.microsoft.com/products/ai-model-catalog https://azure.microsoft.com/products/ai-services/openai-service -
Using LLMs locally https://ollama.com/ Ollama is a tool for easily
running local LLMs on your computer. You can also run it from GitHub Codespaces: aka.ms/ollama-python: Ollama Python Playground -
Using LLMs from Azure OpenAI POST https://SERVICE_NAME.openai.azure.com/openai/deployments/ DEPLOYMENT_NAME/chat/completions?api-version=2024-02-15-preview Authorization: Bearer
AUTH_TOKEN Content-Type: application/json { "messages": [{"role":"system","content":"You are an AI assistant that loves emojis."}, {"role":"user","content":"What is the capital of France?"}], "max_tokens": 800, "temperature": 0.7, "frequency_penalty": 0, "presence_penalty": 0, "top_p": 0.95, "stop": null } https://github.com/pamelafox/python-openai-demos -
Integrating domain knowledge Fine tuning Learn new skills (permanently) Retrieval
Augmented Generation Learn new facts (temporarily) High cost, time -
RAG: Retrieval Augmented Generation Document Search vehicle | year |
msrp | acceleration | --- | --- | --- | --- | --- | --- Prius (1st Gen) | 1997 | 24509.74 | 7.46 | Prius (2nd Gen) | 2000 | 26832.25 | 7.97 | Prius (3rd Gen) | 2009 | 24641.18 | 9.6 | Prius V | 2011 | 27272.28 | 9.51 | Prius C | 2012 | 19006.62 | 9.35 | Prius PHV | 2012 | 32095.61 | 8.82 | Prius C | 2013 | 19080.0 | 8.7 | Prius | 2013 | 24200.0 | 10.2 | Prius Plug-in | 2013 | 32000.0 | 9.17 | Large Language Model The Prius V has an acceleration of 9.51 seconds from 0 to 60 mph. User Question How fast is the Prius V? -
RAG with Azure OpenAI POST https://SERVICE_NAME.openai.azure.com/openai/deployments/ DEPLOYMENT_NAME/chat/completions?api-version=2024-02-15-preview Authorization: Bearer {{$dotenv
TOKEN}} Content-Type: application/json {"messages": [ {"role":"system", "content":"You are a helpful assistant that answers questions about cars based off a hybrid car data set. You must use the data set to answer the questions, you should not provide any info that is not in the provided sources. Sources are provided as a Markdown table." }, {"role":"user", "content": "How fast is the Prius V?\n\nSources: vehicle | year | msrp | acceleration | mpg | class\n --- | --- | --- | --- | --- | --- |\nPrius (1st Gen) | 1997 | 24509.74 | 7.46 | 41.26 | Compact|\nPrius (2nd Gen) | 2000 | 26832.25 | 7.97... }]} https://github.com/pamelafox/python-openai-demos/blob/main/retrieval_augmented_generation.py -
RAG in the wild GitHub Copilot (RAG on your VSCode
workspace) Teams Copilot (RAG on your chats) Bing Copilot (RAG on the web) -
Types of RAG RAG flows: • Simple RAG • Advanced
RAG with Query rewriting RAG sources: • Structured data (DB tables) • Documents (PDF, MD, etc.) • GraphRAG -
Simple RAG Large Language Model For great hiking shoes, consider
the TrekExtreme Hiking Shoes 1 or the Trailblaze Steel-Blue Hiking Shoes 2 User Question What's the best shoe for hiking? Search [101]: Name: TrekExtreme Hiking Shoes Price: 135.99 Brand: Raptor Elite Type: Footwear Description: The Trek Extreme hiking shoes by Raptor Elite are built to ensure any trail. … -
Simple RAG (on PostgreSQL) Azure OpenAI + Azure PostgreSQL Flexible
Server + Azure Container Apps Code: aka.ms/rag-postgres Demo: aka.ms/rag-postgres/demo -
Advanced RAG with query re-writing Large Language Model For great
hiking shoes, consider the TrekExtreme Hiking Shoes 1 or the Trailblaze Steel-Blue Hiking Shoes 2 User Question what's a good shoe for a mountain trale? Search [101]: Name: TrekExtreme Hiking Shoes Price: 135.99 Brand: Raptor Elite Type: Footwear Description: The Trek Extreme hiking shoes by Raptor Elite are built to ensure any trail. … mountain trail shoe Large Language Model -
Query rewriting with function calling User Question Do you sell
climbing gear cheaper than $30? Large Language Model with function calling search_database( "climbing_gear", {"column": "price", "operator" : "<", "value" : "30" } ) -
Advanced RAG with query rewriting via function calling LLM We
offer 2 climbing bags for your budget: SummitStone Chalk Bag 1 Guardian Blue Chalk Bag 2 User Question Do you sell climbing gear cheaper than $30? “Do you sell…” [12]: Name: SummitStone Chalk Bag Price:29.99 Brand:Grolltex Type:Climbing Description: The SummitStone Chalk Bag in forest green is a must- have for climbers seeking adventure. … Search price < 30 LLM with function calling “Do you sell…” “climbing gear” -
RAG on documents Search PerksPlus.pdf#page=2: Some of the lessons covered
under PerksPlus include: · Skiing and snowboarding lessons · Scuba diving lessons · Surfing lessons · Horseback riding lessons These lessons provide employees with the opportunity to try new things, challenge themselves, and improve their physical skills.…. Large Language Model Yes, your company perks cover underwater activities such as scuba diving lessons 1 User Question Do my company perks cover underwater activities? -
RAG on documents Azure OpenAI + Azure AI Search +
Azure App Service Supports simple and advanced flows (Ask tab vs. Chat tab) Code: aka.ms/ragchat Demo: aka.ms/ragchat/demo -
RAG data sources Documents (Unstructured data) PDFs, docx, pptx, md,
html, images You need an ingestion process for extracting, splitting, vectorizing, and storing document chunks. Azure AI Search + Integrated Vectorization Document Intelligence or Database rows (Structured data) You need a way to vectorize & search target columns. • PostgreSQL + pgvector • Azure SQL + vector search (upcoming) • Cosmos MongoDB + vector -
GitHub Copilot for Azure (@azure) https://aka.ms/GitHubCopilotForAzure • Helps developers in
VS Code learn about Azure services and how to use, deploy and manage them find and deploy application templates quickly and easily to Azure diagnose and troubleshoot application issues work independently, minimizing reliance on senior team members and devops -
Architecture Template Recommender Service Large Language Model @azure help me
build a python- powered RAG app [{"name": "Chat with... "url": "https://github... ... }] User Prompt + Function definitions "Call Template Recommender" Orchestrator 1 2 3 -
RAG components Component Examples Retriever: A knowledge base that can
efficiently retrieve sources that match a user query (Ideally supports both vector and full-text search) Azure AI Search, Azure CosmosDB, PostgreSQL, Qdrant, Pinecone LLM: A model that can answer questions based on the query based on the provided sources, and can include citations OpenAI: GPT 3.5, GPT 4, GPT-4o Azure AI Studio: Meta Llama3, Mistral, Cohere R+ Orchestrator (optional): A way to organize calls to the retriever and LLM Community: Langchain, Llamaindex Microsoft: Semantic Kernel, Promptflow Features Chat history, Feedback buttons, Text-to-speech, User login, File upload, Access control, etc. -
Many ways to build a RAG app No Code Copilot
studio Azure Studio On Your Data Open source solutions High Code Low Code -
Copilot Studio Retriever: Uploaded files LLM: GPT 3.5, GPT 4
https://copilotstudio.preview.microsoft.com/ -
Azure OpenAI Studio – On Your Data https://learn.microsoft.com/azure/ai-services/openai/concepts/use-your-data Retriever: Azure
AI Search Azure Blob Storage Azure CosmosDB for MongoDB vCore URL/Web address Uploaded files LLM: GPT 3.5/4 Features: User authentication Chat history persistence -
Open-source RAG chat app: Python Retriever: Azure AI Search LLM:
GPT 3.5/4 Features: Multi-turn chats User authentication with ACLs Chat with image documents github.com/Azure-Samples/azure-search-openai-demo "Building RAG apps in Python" September 3, 10:00 PM UTC / 03:00 PM PT aka.ms/raghack/python -
Open-source RAG chat app: .NET Retriever: Azure AI Search LLM:
GPT 3.5/4 Orchestrator: Semantic Kernel Features: Voice chat github.com/Azure-Samples/azure-search-openai-demo-csharp "Building RAG apps in .NET" Sept. 3, 6:00 PM UTC / 11:30 AM PT aka.ms/raghack/dotnet -
Open-source RAG chat app: JavaScript Retriever: Cosmos DB for NoSQL
LLM: GPT 3.5/4 Orchestrator: Langchain.JS Features: Serverless architecture (Azure Functions) https://github.com/Azure-Samples/serverless-chat-langchainjs "Building RAG apps in Langchain.JS" September 4, 3:00 PM UTC / 8:00 AM PT aka.ms/raghack/js -
Open-source RAG chat app: Java Retriever: Azure AI Search LLM:
GPT 3.5/4 Orchestrator: Semantic Kernel https://github.com/Azure-Samples/azure-search-openai-demo-java/ "Building RAG apps with Java + Semantic Kernel" Sept. 9, 3:00 PM UTC / 8:00 AM PT aka.ms/raghack/java-sk -
Open-source RAG chat solution: Azure AI Studio Retriever: Azure AI
Search LLM: GPT 3.5/4 Orchestrator: PromptFlow Features: CosmosDB user info lookup https://github.com/Azure-Samples/contoso-chat "Building RAG apps with Azure AI Studio" Sept. 3, 8:00 PM UTC / 1:00 PM PT aka.ms/raghack/aistudio -
RAGHack: Next steps Register for the hackathon @ aka.ms/raghack/register Hack,
hack, hack! Try our code samples or start from scratch! For any questions, post in GitHub forum or attend Discord Office Hours. Join more live RAG sessions! aka.ms/raghack/streams Submit your project before September 16th 11:59 PM PT to win prizes! aka.ms/raghack