Upgrade to Pro — share decks privately, control downloads, hide ads and more …

FrankenJS: Large Language Models, Daten & APIs:...

FrankenJS: Large Language Models, Daten & APIs: Integration von Generative AI in eigene Anwendungen

Menschliche Sprache als Universal Interface für Software-Lösungen - hört sich spannend an! Jenseits des ChatGPT-Hypes taucht Christian in die Welt der Large Language Models (LLMs), Daten und APIs ein und konzentriert sich darauf, wie man AI-Funktionalität sinnvoll in eigene Anwendungen integrieren kann. Wir werden pragmatische Szenarien und Use Cases untersuchen, die das Potenzial von LLMs (wie GPT oder Llama) demonstrieren - und erörtern, wie AI-Techniken in bestehende Architekturen einbezogen werden können. Die Teilnehmer erhalten erste Einblicke in Frameworks wie LangChain aus der Python-Welt zur Programmierung LLM-basierter Systeme. Zudem werden wir darauf eingehen, nicht nur Closed-Source-Systeme (wie OpenAI) zu nutzen, sondern auch Open-Source-Optionen (wie Llama) in Betracht zu ziehen, um unterschiedlichen Anforderungen gerecht werden zu können. (Und nein, dieser Abstract wurde nicht von ChatGPT geschrieben.)

Christian Weyer

January 30, 2024
Tweet

More Decks by Christian Weyer

Other Decks in Programming

Transcript

  1. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Christian Weyer @christianweyer CTO, Technology Catalyst
  2. § Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures

    § Microsoft Regional Director § Microsoft MVP for Developer Technologies & Azure ASPInsider, AzureInsider § Google GDE for Web Technologies [email protected] @christianweyer https://www.thinktecture.com Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Christian Weyer Co-Founder & CTO @ Thinktecture AG 2
  3. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Our journey 3 AI all-the- things? Integrating LLMs Selected Scenarios Exciting Times… Democratizing Generative AI
  4. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen AI all-the-things? 4
  5. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen AI all-the-things? 5 Data Science Artificial Intelligence Machine Learning Unsupervised, supervised, reinforcement learning Deep Learning ANN, CNN, RNN etc. NLP Generative AI GAN, VAE, Transformers etc. Image / Video Generation GAN, VAE Large Language Models Transformers
  6. § LLMs generate text based on input § LLMs can

    understand text – this changes a lot § Prompts are the universal interface (“UI”) → unstructured text with semantics § Human language evolves as a first-class citizen in software architecture 🤯 Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Large Language Models (LLMs) 6 Text… – really, just text?
  7. § LLMs are programs § LLMs are highly specialized neural

    networks § LLMs use(d) lots of data § LLMs need a lot of resources to be operated § LLMs have an API to be used through Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Large Language Models demystified 7
  8. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Using LLMs: It’s just APIs ! Inference, FTW. 9
  9. GPT-4 API access via OpenAI Playground Large Language Models, Daten

    & APIs Integration von Generative AI in eigene Anwendungen DEMO 10
  10. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen 11 Choosing a framework for building LLM-based applications https://trends.google.com/trends/explore?q=LangChain,LlamaIndex,HayStack,Semantic%20Kernel&hl=en
  11. § OSS framework for developing applications powered by LLMs §

    > 1000 contributors § Python and Typescript versions § Chains for sequences of LLM-related actions in code § Abstractions for § Prompts & LLMs (local and remote) § Memory § Vector stores § Tools § Loading text from a wide range of sources Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen LangChain - building LLM-based applications 12
  12. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Selected Scenarios 13
  13. Text generation § LLMs are good in generating text §

    Regular text § Code § SQL (beware!) § JSON § etc. Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Typical LLM scenarios: 14
  14. Extracting meaning in text § LLM can be instructed to,

    e.g. § Do sentiment analysis § Extract information from text § Extracting structured information § JSON, TypeScript types, etc. § Via tools like Kor, TypeChat, or Open AI Function/Tool Calling Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Typical LLM scenarios: 15
  15. Extracting structured data (LangChain + Kor) Large Language Models, Daten

    & APIs Integration von Generative AI in eigene Anwendungen DEMO 16
  16. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Answering Questions on Data - Retrieval-augmented generation (RAG) Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Text Question Answer LLM 17 Vector DB Embedding model Embedding model 💡 Indexing / Embedding QA
  17. Learning about my company’s policies via Slack (LangChain) Large Language

    Models, Daten & APIs Integration von Generative AI in eigene Anwendungen DEMO 18
  18. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen Democratizing Generative AI 19
  19. Large Language Models, Daten & APIs Integration von Generative AI

    in eigene Anwendungen LLMs everywhere OpenAI-related (cloud) OpenAI Azure OpenAI Service Big cloud providers Google Model Garden on Vertex AI Amazon Bedrock Other providers Antrophic Cohere HuggingFace … Open-source Edge IoT Server Desktop Mobile Web Open-source 20
  20. Local RAG with Mistral OSS LLM (llama.cpp & LM Studio)

    Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen DEMO 21
  21. § LLMs enable new scenarios & use cases to incorporate

    human language into software solutions § Fast moving and changing field § Every week something “big” happens in LLM space § Frameworks & ecosystem are evolving together with LLMs § Closed vs open LLMs § Competition drives invention & advancement § SISO (sh*t in, sh*t out) § Quality of results heavily depends on your data & input Large Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Current state 23
  22. Potential for LLM-AI-powered human-machine workflows via universal interface agents Large

    Language Models, Daten & APIs Integration von Generative AI in eigene Anwendungen Outlook 24
  23. Thank you! Christian Weyer https://thinktecture.com/christian-weyer 25 Selected demos: Extract structured

    information: https://github.com/thinktecture-labs/llm-extract-structured-information-langchain-kor Local RAG with PDFs: https://github.com/thinktecture-labs/rag-chat-with-pdf-local-llm