Upgrade to Pro — share decks privately, control downloads, hide ads and more …

summit Community Days: Large Language Models, D...

summit Community Days: Large Language Models, Daten & APIs: Integration von Generative AI in eigene Anwendungen

Jenseits des ChatGPT-Hypes taucht Christian in die Welt der Large Language Models (LLMs), Daten und APIs ein und konzentriert sich darauf, wie Sie AI-Funktionalität sinnvoll in Ihre Anwendungen integrieren können. Wir werden pragmatische Szenarien und Use Cases untersuchen, die das Potenzial von LLMs (wie GPT oder Llama) demonstrieren - und erörtern, wie AI-Techniken in bestehende Architekturen einbezogen werden können. Die Teilnehmer erhalten erste Einblicke in LangChain als Python-Toolkit. Zudem werden wir darauf eingehen, nicht nur Closed-Source-Systeme (wie OpenAI) zu nutzen, sondern auch Open-Source-Optionen in Betracht zu ziehen, um unterschiedlichen Anforderungen gerecht werden zu können.

Christian Weyer

May 03, 2024
Tweet

More Decks by Christian Weyer

Other Decks in Programming

Transcript

  1. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Christian Weyer Co-Founder & CTO @christianweyer Community Days Softwarearchitektur & Softwareentwicklung
  2. § Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures

    § Microsoft Regional Director § Microsoft MVP for Developer Technologies & Azure ASPInsider, AzureInsider § Google GDE for Web Technologies [email protected] @christianweyer https://www.thinktecture.com Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Christian Weyer Co-Founder & CTO @ Thinktecture AG 2
  3. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Our journey today 3 AI all-the- things? LLMs in your Solutions Talk to your Data Exciting Times… Democratizing Generative AI Talk to your Systems
  4. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen AI all-the-things? 4
  5. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen AI all-the-things? 6 Data Science Artificial Intelligence Machine Learning Unsupervised, supervised, reinforcement learning Deep Learning ANN, CNN, RNN etc. NLP Generative AI GAN, VAE, Transformers etc. Image / Video Generation GAN, VAE Large Language Models Transformers
  6. § LLMs generate text based on input § LLMs can

    understand text – this changes a lot § Without having to train them on data, domains, or use cases § Prompts are the universal interface (“UI”): unstructured text with semantics as input § Human language evolves as a first-class citizen in software architecture 🤯 Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Large Language Models (LLMs) 7 Text… – really, just text?
  7. § LLMs are programs § LLMs are highly specialized neural

    networks § LLMs use(d) lots of data § LLMs need a lot of resources to be operated § LLMs have an API to be used through Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Large Language Models demystified 8
  8. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen LLMs in your Solutions 9
  9. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Using LLMs: It’s just HTTP APIs Inference, FTW. 10
  10. GPT API access Open AI Playground Large Language Models, Daten

    & APIs Integration von Generative AI in eigene Anwendungen DEMO 11
  11. Barebones SDKs § E.g. Open AI SDK § Available for

    any programming language § Basic abstraction over HTTP APIs § Also available from other providers LangChain § Large community § Python and Typescript versions § Chains for sequences of LLM-related actions in code § Abstractions for § Prompts & LLMs § Memory § Vector stores § Tools § Loading text from a wide range of sources Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Building LLM-based end-to-end applications 12
  12. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Talk to your Data 13
  13. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Answering Questions on Data Retrieval-augmented generation (RAG) Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Text Question Answer LLM 14 Embedding model Embedding model 💡 Indexing / Embedding Question Answering Vector DB
  14. RAG: Learning about company’s policies via Slack LangChain, Weaviate –

    Mixtral-8x7B on Groq Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen DEMO 15
  15. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Talk to your Systems 16
  16. § Clever & strict prompting § Schema description: JSON, TypeScript

    types, etc. § Framework or tools support § Kor, TypeChat, etc. § Open AI Function Calling Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Extract structured data from textual information 17
  17. Extracting structured data from text: Flight search OpenAI Function Calling,

    LangChain, FastAPI, JS - GPT-4-Turbo Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen DEMO 18
  18. Extracting structured data from voice: Form filling JSON extraction, Angular

    Forms - Mixtral-8x7B on Groq Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen DEMO 19
  19. § Integrate LLM-external systems to aid LLMs § Tool /

    function calling standard established by OpenAI § LLM outputs JSON containing arguments to call one or many functions § LLM does not call the function § All major libs support tool calling § OpenAI SDKs § LangChain § Semantic Kernel § etc. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Extending LLM capabilities 20 curl https://api.openai.com/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-3.5-turbo", "messages": [ { "role": "user", "content": "What is the weather like in Boston?" } ], "tools": [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] } }, "required": ["location"] } } } ], "tool_choice": "auto" }'
  20. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Talking to internal APIs – Ask for experts availability 21 Angular PWA Open AI Speech-to-Text Internal Systems Gateway Open AI GPT-4-T Open AI Text-to-Speech Transcribe spoken text Transcribed text Check for experts availability with text Extract { experts, booking times } from text Structured JSON data (Tool calling) Generate response with availability Response Response with experts availability 🗣 🔉 Speech-to-text for response Response audio Internal Company API Query Availability API Availability When is CL…? CL will be…
  21. Ask for experts availability in my company systems Angular, Speech-to-text,

    internal HTTP API, node.js OpenAI SDK + Tool Calling, Text-to-speech – GPT-4-Turbo Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen DEMO 22
  22. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Democratizing Generative AI 23
  23. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen LLMs everywhere OpenAI-related (cloud) OpenAI Azure OpenAI Service Big cloud providers Google Model Garden on Vertex AI Amazon Bedrock Open-source Edge IoT Server Desktop Mobile Web 24 Other providers Antrophic Cohere Mistral AI Hugging Face Open-source
  24. § Open-source community drives innovation in Generative AI § Important

    factors § Use case § Parameter size § Quantization § Processing power needed § Mistral-based family shows big potential for local use cases (7B params) Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Open-source LLMs thrive 25
  25. Local RAG: Mistral-7B open-source LLM llama.cpp, ollama, LangChain, StreamLit Large

    Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen DEMO 26
  26. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Exciting Times… 27
  27. § LLMs enable new scenarios & use cases to incorporate

    human language into software solutions § Fast moving and changing field § Every week something “big” happens in LLM space § Frameworks & ecosystem are evolving together with LLMs § Closed vs open LLMs § Competition drives invention & advancement § SLMs: specialized, fine-tuned for domains § SISO (sh*t in, sh*t out) § Quality of results heavily depends on your data & input Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Current state 28
  28. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen 2024: The year of SLMs & CPU inference 29