AI-driven Software Development with Azure AI

Transcript

1. AI-DRIVEN SOFTWARE DEVELOPMENT WITH AZURE AI Jörg Neumann
2. TOPICS  AI  Frontend Technologies  Mobile Development  Consulting, Coaching, Training CONTACT  Mail: joerg.neumann@neogeeks.de  Twitter: @JoergNeumann  Web: www.neogeeks.de JÖRG NEUMANN Founder & CEO NeoGeeks GmbH Azure OpenAI Service, Azure Machine Learning Platform
3. AI-SCENARIOS POPULAR CASES  Chat with own data  Customer support  Call centers  …
4. RETRIEVAL AUGMENTED GENERATION (RAG) GOAL  Extend the model with external data sources  Integrate domain-specific knowledge  Model doesn’t need to be trained
5. RETRIEVAL AUGMENTED GENERATION question query own data add results to the prompt query model answer
6. RETRIEVAL ARGUMENTED GENERATION (RAG) Chunk Chunk Chunk Chunk Embeddings Embeddings Embeddings Embeddings Vector Database PDF Document user search Embeddings Search answer LLM weight results
7. PROMPT ENGINEERING Answer the question based on the context provided below. If the question cannot be answered with the provided information, respond with 'I don't know’. context: Large Language Models (LLMs) are the latest models used in NLP. Their superior performance over smaller models has made them incredibly useful for developers. These models can be accessed via the 'transformers' library by Hugging Face, via OpenAI with the 'openai' library, and via Cohere with the 'cohere' library. question: Which libraries and model providers offer LLMs?“ instructions prompt external information Which libraries and model providers offer LLMs?
8. IMPLEMENTATION CHALLENGES  Tables and illustrations in documents  Answers have no source citations  Answers can be outdated  The model can hallucinate  Limit context to specific data  Content filters
9. IMPLEMENTATION CODING  Python, C#, Java FRAMEWORKS  LangChain, LlamaIndex, …  Semantic Kernel  Spring AI VECTOR STORES  Pinecone, Chroma, Milvus, Qdrant, …
10. AI IN AZURE AZURE OPENAI SERVICE  Models from OpenAI  Endpoint for API access  Available models depend on region AZURE OPENAI STUDIO  Deployment and configuration of the models
11. Documents Table Storage Blob Storage AI Search OpenAI Service Database Data App ARCHITECTURE
12. DEMO Azure OpenAI Studio Source: https://oai.azure.com
13. AI IN AZURE AZURE AI STUDIO  Development of complex AI solutions  Multi-Model Support  Prompt Flow  Debugging & Metrics  https://ai.azure.com
14. DEMO Azure AI Source: https://oai.azure.com
15. CONCLUSION  RAG is a complex pattern  Numerous frameworks and tools are required  Azure OpenAI and Azure AI Search offers an easy low-code solution for RAG scenarios  Complex scenarios can be implemented with Azure AI Studio
16. Q& A