AI Development in .NET - Microsoft.Extensions.AI

Devnot - Dotnet Konferansı 2025
24.05.2025
Sheraton Grand Ataşehir, İstanbul

Transcript

1. AI Development in .NET Mert Metin #dotnetkonf25 Microsoft.Extensions.AI
2. Interests Microsoft / .NET software architecture, secure software, clean code infrastructure, system design Senior Software Engineer Blogger, speaker MERT METİN QR to reach me
3. Agenda Basic Concepts of GenAI Exploring the .NET Package - Microsoft.Extensions.AI
4. Brief Recap of the GenAI Concepts Generative AI, LLM, Prompt, Roles, RAG ,Agent, Token, Embedding, Tools
5. Generative AI Subset of AI Creates new content such as text, code, audio, image based on user prompts and learned data.
6. Large Language Model LLMs are optimized for complex tasks, trained on huge datasets, and capable of understanding and interpreting human language.
7. Prompt An input that is used to communicate with the model to generate desired output. Prompt Engineering is the practice of designing effective prompts. Prompts must be; clear, specific, goal oriented, have contextual information
8. System Specify “How will the model behave?” Roles User Prompt and query by user. Asistant Model responds using user’s prompt. It is used to ensure focus on a specific task or behavior pattern.
9. Agents They are capable of reasoning, planning, and interacting with their environment, using generative AI to perform tasks autonomously.
10. Tool Functions provided to LLMs must have clear objectives. Tools enable agents to perform actions such as web search, retrieving information from external sources and API calls.
11. Token Meaningful unit processed by LLM. Token is not only a word. tokenized by gpt-4o
12. Embedding - Vector Database Embedding is an operation that converts text, images into numerical data (vectors) that machines can understand. A vector database stores numerical data as vectors, enabling efficient similarity search
13. Retrieval Augmented Generation (RAG) Enhances LLMs with external data sources (databases, documents, APIs). Improves response relevance and accuracy.
14. Exploring the .NET Package Microsoft.Extensions.AI Provides utilities for working with generative AI components.
15. Why Support for switching between different LLM providers Manageable, Simple, and Rapid Development Provide implementations for dependency injection, caching, telemetry and other cross-cutting concerns. Implement chat and embedding features using any LLM provider.
16. IChatClient Connecting to LLM providers such as Open AI, Ollama, Azure Open AI
17. Microsoft.Extensions.AI.OpenAI LLM provider for OpenAI Working With Model Providers
18. Microsoft.Extensions.AI.Ollama LLM provider for Ollama Useful for working on local machine Working With Model Providers Note: This package is deprecated and the OllamaSharp package is recommended.
19. OllamaSharp The eaisest way to use the Ollama in .NET Working With Model Providers
20. GetResponseAsync Sends a user chat text message and returns the response messages. Return type is ChatResponse which represents the response to a chat request.
21. GetStreamingResponseAsync Sends chat messages and streams the response. Return type is IAsyncEnumerable<ChatResponseUpdate> which provides a stream of updated chat response.
22. Case - Sentiment Analysis Customer’s order reviews of the restaurant
23. Output - GetResponseAsync
24. Output - GetStreamingResponseAsync
25. Conversation Application needs to keep conversation context. So, send conversation history to the model using roles. Assistant role is set by AddMessages. Case - git commit generator
26. History and Roles ChatMessage list often represents the history of whole chat messages that are part of the conversation. Chat result
27. Tool Calling UseFunctionInvocation - set method name as delegate type
28. Tool
29. Output Usage Informations Response
30. Embeddings GenerateEmbeddingAsync has more detailed information about embeddings and Vector collection is also included. GenerateEmbeddingVectorAsync returns the vector collection.
31. Case - RelatedNews
32. Output
33. Dependency Injection Inject as ChatClient, EmbeddingClient OpenAI Ollama
34. Dependency Injection Call from API Controller
35. Dependency Injection Using Keyed Services
36. Caching Caching Alternatives: Memory or Redis
37. Caching UseDistributedCache pipeline for caching in IChatClient
38. Redis Cache
39. Open Telemetry
40. Jaeger
42. thanks for listening To reach me To get slide