BASTA! 2024 - Session: Large Language Models: Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action

Menschliche Sprache als Universal Interface (UI) für Software-Lösungen – geschrieben oder gar gesprochen. Hört sich spannend an! In dieser Session taucht Christian Weyer in die Welt der Large Language Models (LLMs) ein und konzentriert sich darauf, wie man Generative-AI auf Basis von LLMs über Daten und APIs sinnvoll in eigene Anwendungslösungen integrieren kann. Er stellt pragmatische Use Cases und Patterns vor, die das Potenzial von LLMs wie OpenAI GPT oder Open-Source-Varianten wie Llama und Mistral demonstrieren. Begeben wir uns gemeinsam auf einen möglichen Weg zu 'User Interfaces' jenseits von reinen GUIs & bunti-bunti. Uuund Action!

Transcript

1. Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action Christian Weyer | Co-Founder & CTO | Thinktecture AG
2. § Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures § Microsoft Regional Director § Microsoft MVP for Developer Technologies & Azure ASPInsider, AzureInsider § Google GDE for Web Technologies christian.weyer@thinktecture.com @christianweyer https://www.thinktecture.com Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action Christian Weyer Co-Founder & CTO @ Thinktecture AG 2
3. The image part with relationship ID rId3 was not found in the file. The image part with relationship ID rId3 was not found in the file. Special Day Generative AI für Business-Anwendungen Thema Sprecher Datum, Uhrzeit Large Language Models: Szenarien, Use Cases & Patterns für Business- Anwendungen - in Action Christian Weyer DI, 17. September 2024, 10.45 bis 11.45 Real-World RAG: Eigene Daten & Dokumente mit semantischer Suche & LLMs erschließen Sebastian Gingter DI, 17. September 2024, 12.15 bis 13.15 Von 0 zu Smart: SPAs mit Generative AI aufwerten Max Marschall DI, 17. September 2024, 15.30 bis 16.30 Deep Dive in OpenAI Hosted Tools Rainer Stropek DI, 17. September 2024, 17.00 bis 18.00
4. Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action Our journey today 4 AI all-the- things? LLMs in your Solutions Talk to your Data Recap & Outlook Generative AI everywhere Talk to your Systems
5. Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action AI all-the-things? 5
6. Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action AI all-the-things? 6 Data Science Artificial Intelligence Machine Learning Unsupervised, supervised, reinforcement learning Deep Learning ANN, CNN, RNN etc. NLP (Natural Language Processing) Generative AI GAN, VAE, Transformers etc. Image / Video Generation GAN, VAE Large Language Models Transformers
7. § LLMs generate text based on input § LLMs can understand text – this changes a lot § Without having to train them on domains or use cases § Prompts are the universal interface (“UI”) → unstructured text with semantics § Human language evolves as a first-class citizen in software architecture 🤯 Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action Large Language Models (LLMs) – like GPT powering ChatGPT 7
8. § LLMs are programs § LLMs are highly specialized neural networks § LLMs use(d) lots of data § LLMs need a lot of resources to be operated § LLMs have an API to be used through Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action Large Language Models demystified 🔍 8
9. Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action LLMs in your Solutions 9
10. § LLMs are always part of end-to-end architectures § Client apps (Web, desktop, mobile) § Services with APIs § Databases § etc. § An LLM is ‘just’ an additional asset in your architecture § Enabling human language understanding & generation § It is not the Holy Grail for everything Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action End-to-end architectures with LLMs 10 Clients Services LLMs Desktop Web Mobile Service A Service B Service C API Gateway Monitoring LLM 1 LLM 2
11. Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action Using LLMs: It’s just HTTP APIs Inference, FTW 11
12. GPT-4 API access OpenAI Playground Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action DEMO 12
13. Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action Generative AI everywhere 13
14. Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action LLMs everywhere OpenAI-related (cloud) OpenAI Azure OpenAI Service Big cloud providers Google Model Garden on Vertex AI Amazon Bedrock Edge 14 Other providers Anthropic Cohere Mistral AI Hugging Face Groq
15. § Open-source community drives innovation in Generative AI § Llama- & Mistral-based families show big potential § Success factors § Use case § Parameter size § Quantization § Processing power needed § CPU optimization on its way Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action Open-weights LLMs thrive 15 § Local inference runtimes with APIs § E.g. llama.cpp, ollama, VLLM § Local UIs § E.g. Open WebUI
16. Local open-source models, APIs & UIs Ollama, Open WebUI Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action DEMO 16
17. Barebones SDKs § E.g. Open AI SDK § Available for any programming language § Basic abstraction over HTTP APIs § Lot of inference runtimes offer Open AI API compatible APIs § Also available from other providers § Mistral § Anthropic § Cohere § Ollama § Etc. Frameworks – e.g. LangChain, Semantic Kernel § Provide abstractions – typically for § Prompts & LLMs § Memory § Vector stores § Tools § Loading data from a wide range of sources Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action Building LLM-based end-to-end applications 17
18. Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action Talk to your Data 18
19. Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action Answering questions on data Retrieval-augmented generation (RAG) Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Text Question Answer LLM 19 Embedding model Embedding model 💡 Indexing / Embedding Question Answering Vector DB
20. RAG: Learning about company’s policies via Slack LangChain, Weaviate – GPT-4o Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action DEMO 20
21. Local RAG: Llama 3.1 open-source LLM llama.cpp, ollama, LangChain, StreamLit Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action DEMO 21
22. Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action Talk to your Systems 22
23. § Write or speak your input → get structured data for your programs & systems § Clever & strict prompting § Schema description: Custom format, JSON, TypeScript types, etc. § Framework or tools support § Pydantic, Instructor, TypeChat, etc. § OpenAI Function / Tool Calling Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action 1. Extract structured data from textual information 23
24. Extracting structured data from text & voice: Form filling Data extraction prompt, OpenAI JS SDK, Angular Forms – Mixtral-8x7B on Groq Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action DEMO 24
25. § Integrate LLM-external systems to aid LLMs § Tool / function calling standard established by OpenAI § LLM outputs JSON conforming to a schema § LLM does not call a function § All major libs support tool calling § OpenAI SDKs § LangChain § Semantic Kernel § etc. § You wire up the logic in your code Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action 2. Extending LLM capabilities 25 curl https://api.openai.com/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-4o", "messages": [ { "role": "user", "content": "What is the weather like in Boston?" } ], "tools": [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] } }, "required": ["location"] } } } ], "tool_choice": "auto" }'
26. Ask for experts availability in my company systems Angular, Speech-to-text, internal HTTP API, node.js OpenAI SDK + Tool Calling, Text-to-speech – GPT-4-turbo Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action DEMO 26
27. Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action Talking to internal APIs – Ask for expert’s availability 27 Angular PWA Open AI Speech-to-Text Internal Systems Gateway Open AI GPT-4 Open AI Text-to-Speech Transcribe spoken text Transcribed text Check for experts availability with text Extract { experts, booking times } from text Structured JSON data (Tool calling) Generate response with availability Response Response with experts availability 🗣 🔉 Speech-to-text for response Response audio Internal Company API Query Availability API Availability When is CL…? CL will be…
28. Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action Exciting Times 🫱 🫲 28
29. § LLMs enable new scenarios & use cases to incorporate human language into software solutions § Fast moving and changing field § Every week something “big” happens in LLM space § Frameworks & ecosystem are evolving together with LLMs § Closed vs open LLMs § Competition drives invention & advancement § SLMs: specialized, fine-tuned for domains § Running local models in production is hard! § SISO (sh*t in, sh*t out) § Quality of results heavily depends on your data & input Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action Current state 29
30. Large Language Models Szenarien, Use Cases & Patterns für Business-Anwendungen - in Action The rise of SLMs & CPU inference 30
31. Thank you! Christian Weyer https://thinktecture.com/christian-weyer 31