Upgrade to Pro — share decks privately, control downloads, hide ads and more …

programmier.bar: Large Language Models, Daten &...

programmier.bar: Large Language Models, Daten & APIs - Integration von Generative AI in eigene Anwendungen

Menschliche Sprache als Universal Interface für Software-Lösungen - hört sich spannend an! Jenseits des ChatGPT-Hypes taucht Christian in die Welt der Large Language Models (LLMs), Daten und APIs ein und konzentriert sich darauf, wie man AI-Funktionalität sinnvoll in eigene Anwendungen integrieren kann. Wir werden pragmatische Szenarien und Use Cases untersuchen, die das Potenzial von LLMs (wie GPT oder Llama) demonstrieren - und erörtern, wie AI-Techniken in bestehende Architekturen einbezogen werden können. Die Teilnehmer erhalten erste Einblicke in Frameworks wie LangChain aus der Python-Welt zur Programmierung LLM-basierter Systeme. Zudem werden wir darauf eingehen, nicht nur Closed-Source-Systeme (wie OpenAI) zu nutzen, sondern auch Open-Source-Optionen (wie Llama) in Betracht zu ziehen, um unterschiedlichen Anforderungen gerecht werden zu können.

Christian Weyer

February 01, 2024
Tweet

More Decks by Christian Weyer

Other Decks in Programming

Transcript

  1. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Christian Weyer @christianweyer CTO, Technology Catalyst
  2. § Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures

    § Microsoft Regional Director § Microsoft MVP for Developer Technologies & Azure ASPInsider, AzureInsider § Google GDE for Web Technologies [email protected] @christianweyer https://www.thinktecture.com Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Christian Weyer Co-Founder & CTO @ Thinktecture AG 2
  3. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Our journey 3 AI all-the- things? Integrating LLMs Selected Scenarios End-to-End Exciting Times… Democratizing Generative AI
  4. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen AI all-the-things? 4
  5. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen AI all-the-things? 5 Data Science Artificial Intelligence Machine Learning Unsupervised, supervised, reinforcement learning Deep Learning ANN, CNN, RNN etc. NLP Generative AI GAN, VAE, Transformers etc. Image / Video Generation GAN, VAE Large Language Models Transformers
  6. § LLMs generate text based on input § LLMs can

    understand text – this changes a lot § Prompts are the universal interface (“UI”) → unstructured text with semantics § Human language evolves as a first-class citizen in software architecture 🤯 Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Large Language Models (LLMs) 6 Text… – really, just text?
  7. § LLMs are programs predicting the next token § LLMs

    are highly specialized neural networks § LLMs use(d) lots of data § LLMs need a lot of resources to be operated § LLMs have an API to be used through Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Large Language Models demystified 7
  8. § LLMs are always part of end-to-end architectures § HTTP/Web/REST

    APIs § Databases § Client apps (Web, desktop, mobile) § etc. § An LLM is ‘just’ an additional asset in your architecture § It is not the Holy Grail for everything! Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen End-to-end architectures with LLMs 9
  9. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Using LLMs: It’s just APIs ! Inference, FTW. 10
  10. GPT-4 API access via OpenAI Playground Large Language Models, Daten

    & APIs Integration von Generative AI in eigene Anwendungen DEMO 11
  11. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen The best tool for developers to talk to LLMs! 12
  12. § OSS framework for developing applications powered by LLMs §

    > 1000 contributors § Python and Typescript versions § Chains for sequences of LLM-related actions in code § Abstractions for § Prompts & LLMs (local and remote) § Memory § Vector stores § Tools § Loading text from a wide range of sources Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen LangChain - building LLM-based applications 13
  13. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Selected Scenarios 14
  14. Text generation § LLMs are good in generating text §

    Regular text § Code § SQL (beware!) § JSON § etc. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Typical LLM scenarios: 15
  15. Extracting meaning in text § LLM can be instructed to,

    e.g. § Do sentiment analysis § Extract information from text § Extracting structured information § JSON, TypeScript types, etc. § Via tools like Kor, TypeChat, or Open AI Function Calling Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Typical LLM scenarios: 16
  16. Extracting structured data (LangChain + Kor) Large Language Models, Daten

    & APIs Integration von Generative AI in eigene Anwendungen DEMO 17
  17. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Answering Questions on Data - Retrieval-augmented generation (RAG) Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Text Question Answer LLM 18 Vector DB Embedding model Embedding model 💡 Indexing / Embedding QA
  18. Learning about my company’s policies via Slack (LangChain) Large Language

    Models, Daten & APIs Integration von Generative AI in eigene Anwendungen DEMO 19
  19. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Democratizing Generative AI 20
  20. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen LLMs everywhere OpenAI-related (cloud) OpenAI Azure OpenAI Service Big cloud providers Google Model Garden on Vertex AI Amazon Bedrock Open-source Edge IoT Server Desktop Mobile Web 21 Other providers Antrophic Cohere Mistral AI Hugging Face Open-source
  21. § Open-source community drives innovation in Generative AI § HuggingFace

    is central place for it § Literally, every week a new and “better” LLM shows up 🤓 § Important factors § Use case § Parameter size § Quantization § Processing power needed § Mistral-based family shows big potential for local use cases (7B params) Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Open-source LLMs thrive 22
  22. Local RAG with Mistral OSS LLM (llama.cpp & LM Studio)

    Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen DEMO 23
  23. Ask for expert availability in my company systems (Speech-to-text, internal

    API, node.js OpenAI SDK, GPT-4, Text-to-speech) Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen DEMO 25
  24. 26 Angular PWA OpenAI Speech-to-Text TT Panorama Gateway OpenAI GPT-4

    OpenAI Text-to-Speech Transcribe spoken text Transcribed text Check for experts availability with text Extract { experts, booking times } from text Structured JSON data Generate response with availability Response Response with experts availability 🗣 🔉 Speech-to-text for response Response audio TT Panorama Query Panorama API Availability Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen
  25. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Exciting Times… 27
  26. § LLMs enable new scenarios & use cases to incorporate

    human language into software solutions § Fast moving and changing field § Every week something “big” happens in LLM space § Frameworks & ecosystem are evolving together with LLMs § Closed vs open LLMs § Competition drives invention & advancement § SISO (sh*t in, sh*t out) § Quality of results heavily depends on your data & input Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Current state 28
  27. Potential for LLM-powered human-machine workflows via universal interface agents Large

    Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Outlook 29
  28. Thank you! Christian Weyer https://thinktecture.com/christian-weyer 30 Selected demos: Extract structured

    information: https://github.com/thinktecture-labs/llm-extract-structured-information-langchain-kor Local RAG with PDFs: https://github.com/thinktecture-labs/rag-chat-with-pdf-local-llm