.NET User Group Switzerland Tour: Large Language Models, Data & APIs: Integrating Generative AI Power into your solutions - with Python & .NET

Let's look beyond the ChatGPT hype! In this session, Christian delves into the world of Large Language Models (LLMs), data, and APIs, focusing on how you can meaningfully integrate AI functionality into your applications. We will explore pragmatic scenarios and use cases that demonstrate the potential of LLMs (such as GPT or Llama) and discuss how AI techniques can be incorporated into existing architectures. Participants will gain initial insights into LangChain as a Python toolkit and Semantic Kernel as a .NET-based SDK. Additionally, we will address the consideration of not only using closed-source systems (like OpenAI) but also taking open-source options into account to meet diverse requirements. Come in and find out!

Transcript

1. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Christian Weyer @christianweyer CTO, Technology Catalyst
2. § Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures § Microsoft Regional Director § Microsoft MVP for Developer Technologies & Azure ASPInsider, AzureInsider § Google GDE for Web Technologies christian.weyer@thinktecture.com @christianweyer https://www.thinktecture.com Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Christian Weyer Co-Founder & CTO @ Thinktecture AG 2
3. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Our journey 3 AI all-the- things? Integrating LLMs Selected Scenarios End-to-End Exciting Times… Democratizing Generative AI
4. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET AI all-the-things? 4
5. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET AI all-the-things? 5 Data Science Artificial Intelligence Machine Learning Unsupervised, supervised, reinforcement learning Deep Learning ANN, CNN, RNN etc. NLP Generative AI GAN, VAE, Transformers etc. Image / Video Generation GAN, VAE Large Language Models Transformers
6. § LLMs generate text based on input § LLMs can understand text – this changes a lot § Prompts are the universal interface (“UI”) → unstructured text with semantics § Human language evolves as a first-class citizen in software architecture 🤯 Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Large Language Models (LLMs) 6 Text… – really, just text?
7. § LLMs are programs § LLMs are highly specialized neural networks § LLMs use(d) lots of data § LLMs need a lot of resources to be operated § LLMs have an API to be used through Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Large Language Models demystified 7
8. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Integrating LLMs 8
9. § LLMs are always part of end-to-end architectures § HTTP/Web/REST APIs § Databases § Client apps (Web, desktop, mobile) § etc. § An LLM is ‘just’ an additional asset in your architecture § It is not the Holy Grail for everything! Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET End-to-end architectures with LLMs 9
10. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Using LLMs: It’s just APIs ! Inference, FTW. 10
11. GPT-4 API access via OpenAI Playground Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET DEMO 11
12. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET The best tool for .NET developers to talk to LLMs! 12 🙈
13. § OSS framework for developing applications powered by LLMs § > 1000 contributors § Python and Typescript versions § Chains for sequences of LLM-related actions in code § Abstractions for § Prompts & LLMs (local and remote) § Memory § Vector stores § Tools § Loading text from a wide range of sources Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET LangChain - building LLM-based applications 13
14. § Microsoft’s OSS framework to integrate LLMs into applications § .NET, Python, and Java versions § .NET 1.0 RTM in December 2023 § Plugins encapsulate AI capabilities § Semantic functions for prompting § Native functions to run local code § Planners are orchestrating LLMS interactions § Not as broad feature set as LangChain § E.g., no concept/abstraction for loading data Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Semantic Kernel - building LLM-based applications 14
15. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Selected Scenarios 15
16. Text generation § LLMs are good in generating text § Regular text § Code § SQL (beware!) § JSON § etc. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Typical LLM scenarios: 16
17. Extracting meaning in text § LLM can be instructed to, e.g. § Do sentiment analysis § Extract information from text § Extracting structured information § JSON, TypeScript types, etc. § Via tools like Kor, TypeChat, or Open AI Functions Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Typical LLM scenarios: 17
18. Extracting structured data (LangChain + Kor) Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET DEMO 18
19. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Answering Questions on Data - Retrieval-augmented generation (RAG) Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Text Question Answer LLM 19 Vector DB Embedding model Embedding model 💡 Indexing / Embedding QA
20. Learning about my company’s policies via Slack (LangChain) Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET DEMO 20
21. Chat with web site documents (Semantic Kernel) Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET DEMO 21
22. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Democratizing Generative AI 22
23. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET LLMs everywhere OpenAI-related (cloud) OpenAI Azure OpenAI Service Big cloud providers Google Model Garden on Vertex AI Amazon Bedrock Other providers Antrophic Cohere HuggingFace … Open-source Edge IoT Server Desktop Mobile Web Open-source 23
24. § Open-source community drives innovation in Generative AI § HuggingFace is central place for it § Literally, every week a new and “better” LLM shows up 🤓 § Important factors § Use case § Parameter size § Quantization § Processing power needed § Mistral-based family shows big potential for local use cases (7B params) Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Open-source LLMs thrive 24
25. Local RAG with Mistral OSS LLM (llama.cpp & LM Studio) Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET DEMO 25
26. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET End-to-End 26
27. Ask for expert availability in my company systems (Speech-to-text, my own API, GPT-4, Text-to-speech) Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET DEMO 27
28. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Exciting Times 28
29. § LLMs enable new scenarios & use cases to incorporate human language into software solutions § Fast moving and changing field § Every week something “big” happens in LLM space § Frameworks & ecosystem are evolving together with LLMs § Closed vs open LLMs § Competition drives invention & advancement § SISO (sh*t in, sh*t out) § Quality of results heavily depends on your data & input Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Current state 29
30. Potential for LLM-powered human-machine workflows via universal interface agents Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Outlook 30
31. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Outlook 31
32. Thank you! Christian Weyer https://thinktecture.com/christian-weyer 32 Selected demos: Extract structured information: https://github.com/thinktecture-labs/llm-extract-structured-information-langchain-kor Local RAG with PDFs: https://github.com/thinktecture-labs/rag-chat-with-pdf-local-llm
33. § Semantic Kernel § https://learn.microsoft.com/en-us/semantic-kernel/overview/ § LangChain § https://www.langchain.com/ § LangChain Agents § https://python.langchain.com/docs/modules/agents/ § ReAct: Synergizing Reasoning and Acting in Language Models § https://react-lm.github.io/ § Prompt Engineering Guide § https://www.promptingguide.ai/ § OpenAI API reference § https://platform.openai.com/docs/api-reference § Azure OpenAI Service REST API reference § https://learn.microsoft.com/en-us/azure/ai-services/openai/reference § Hugging Face Inference Endpoints (for various OSS LLMs) § https://huggingface.co/docs/inference-endpoints/api_reference § Llama2 on Hugging Face § https://huggingface.co/blog/llama2 § OWASP Top 10 for LLM Applications § https://owasp.org/www-project-top-10-for-large-language-model-applications/assets/PDF/OWASP-Top-10-for-LLMs-2023-slides-v1_0_1.pdf Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Links 33