Customizing your RAG Chat App

Slides for talk given at https://github.com/microsoft/AI-Chat-App-Hack

Transcript

3. Customizing your RAG chat app
4. Prerequisites: RAG Chat App This talk will be all about customizing this app: https://github.com/Azure-Samples/azure-search-openai-demo Before you can customize it, you must deploy all the resources. See this doc for tips on deploying for free: aka.ms/ragchat/free (Keep watching either way, you can always rewatch after you deploy!)
5. Today’s topics •Local development tips ( reload) •Customizing the frontend (UI) •Customizing the backend •Ingesting custom data
6. Local development
7. Starting the local server cd app ./start.sh 1. Packages the React frontend typescript files using vite 2. Copies the JS into app/backend folder 3. Starts the Python app in reload mode (which only watches Python files!)
8. Customizing the frontend
9. Hot reloading for frontend files cd app/frontend npm run dev In a new terminal: Then navigate to the URL shown in the output. The backend will still proxy to the previously started Python server.
10. Frontend components Change this file: To customize: app/frontend/index.html title, metadata, script tag app/frontend/public/favicon.ico browser tab icon app/frontend/src/pages/layout/Layout.tsx Navigation bar, colors app/frontend/src/pages/chat/Chat.tsx “Chat” tab and default settings app/frontend/src/pages/ask/Ask.tsx “Ask” tab and default settings
11. Customizing the backend
12. Backend components Change this file: To customize: app/backend/app.py additional routes, app configuration app/backend/approaches/chatreadretrieveread.py “Chat” tab, RAG prompt and flow app/backend/approaches/chatreadretrievereadvision.py “Chat” tab, RAG flow when using vision app/backend/approaches/retrievethenread.py “Ask” tab, RAG prompt and flow app/backend/approaches/retrievethenreadvision.py “Ask” tab, RAG flow when using vision
13. Changing the prompt Assistant helps the company employees with their healthcare plan questions, and questions about the employee handbook. Be brief in your answers. Answer ONLY with the facts listed in the list of sources below. If there isn't enough information below, say you don't know. Do not generate answers that don't use the sources below. If asking a clarifying question to the user would help, ask the question. For tabular information return it as an html table. Do not return markdown format. If the question is not in English, answer in the language used in the question. Each source has a name followed by colon and the actual information, always include the source name for each fact you use in the response. Use square brackets to reference the source, for example [info1.txt]. Don't combine sources, list each source separately, for example [info1.txt][info2.pdf].
14. VS code debugger for backend Config in launch.json
15. Ingesting custom data
16. Removing the current data ./scripts/prepdocs.py --removeall In scripts/prepdocs.sh or scripts/prepdocs.ps1: You can also delete the index entirely in the Portal, if you need to add more fields. Data ingestion guide: Removing documents aka.ms/ragchat/remove-data
17. Adding new data: PDFs 1. Add PDFs to data/ folder 2. Rerun ./scripts/prepdocs.ps1 or ./scripts/prepdocs.sh 3. Script will only process files that don’t yet have a corresponding .md5 file Data ingestion guide: Adding documents aka.ms/ragchat/add-data
18. Adding new data: non-PDFs Options: •Convert data to PDFs (e.g. webpages to PDFs) •Use Document Intelligence to convert •Use integrated vectorization (cloud-based) •Write your own parser / use Llamaindex
19. Converting webpages to PDF 1. Create a list of URLs that need indexing 2. Use Playwright to visit each page & save PDF 3. Add new PDFs as usual 4. Customize frontend to render webpage export function getCitationFilePath(citation: string): string { return `https://playwright.dev/python/docs/${citation}`; } Converting HTML pages to PDFs with Playwright aka.ms/html2pdf
20. Writing a custom parser Demo by Chris Ayers PR: Adding a JSONParser class aka.ms/ragchat/json
21. Integrated vectorization End-to-end data processing tailored to RAG Data source access • Blob Storage • ADLSv2 • SQL DB • CosmosDB • … + Incremental change tracking File format cracking • PDFs • Office documents • JSON files • … + Extract images and text, OCR as needed Chunking • Split text into passages • Propagate document metadata Vectorization • Turn chunks into vectors • OpenAI embeddings or your custom model Indexing • Document index • Chunk index • Both In preview Integrated data chunking and embedding in Azure AI Search aka.ms/integrated-vectorization
22. Adding new features
23. Don’t reinvent the wheel! 1. Check the pull requests (doc upload UI, markdown support, speech, etc) 2. Check other RAG chat repos (collaboration mode, conversation history, thumbs up, etc) 3. Ask in the discussion forum But if you do come up with a new feature, please share!
24. Next steps • Register for the hackathon → • Introduce yourself in our discussion forum • Deploy the repo with the sample data • See steps on low cost deployment → • Start customizing the project! • Post in forum if you have any issues deploying or questions about customization. • Join tomorrow’s session: Azure AI Search Best Practices aka.ms/hacktogether/chatapp aka.ms/ragchat/free