Upgrade to Pro — share decks privately, control downloads, hide ads and more …

JAX 2024: AI++: Integration von Large Language ...

JAX 2024: AI++: Integration von Large Language Models (LLMs) in moderne Business-Software – Sprachverständnis ändert alles

Menschliche Sprache als Universal Interface für Software-Lösungen - hört sich spannend an! Und das Ganze gemischt mit Bildern verspricht eine neue Art des Benutzerzugangs zu Anwendungen. Jenseits des ChatGPT-Hypes taucht Christian Weyer in die Welt der Large Language Models (LLMs) ein und konzentriert sich darauf, wie man Generative-AI-Funktionalität über Daten und APIs sinnvoll in eigene Applikationen integrieren kann. Wir werden pragmatische Szenarien und Use Cases untersuchen, die das Potenzial von LLMs (GPT- oder Llama-basiert) demonstrieren - und erörtern, wie AI-Techniken in bestehende Architekturen einbezogen werden können. Wir werden auch darauf eingehen, nicht nur Closed-Source-Lösungen (wie OpenAI) zu nutzen, sondern Open-Source-Optionen (wie Llama oder Mistral) in Betracht zu ziehen, um unterschiedlichen Anforderungen gerecht werden zu können. Universal User Interfaces jenseits von Bunti-bunti - come in and find out!

Christian Weyer

April 24, 2024
Tweet

More Decks by Christian Weyer

Other Decks in Programming

Transcript

  1. AI++: Integration von Large Language Models (LLMs) in moderne Business-Software

    – Sprachverständnis ändert alles Christian Weyer @christianweyer CTO, Technology Catalyst
  2. § Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures

    § Microsoft Regional Director § Microsoft MVP for Developer Technologies & Azure ASPInsider, AzureInsider § Google GDE for Web Technologies [email protected] @christianweyer https://www.thinktecture.com AI++: Integration von Large Language Models (LLMs) in moderne Business-Software Sprachverständnis ändert alles Christian Weyer Co-Founder & CTO @ Thinktecture AG 2
  3. AI++: Integration von Large Language Models (LLMs) in moderne Business-Software

    Sprachverständnis ändert alles Our journey today 3 AI all-the- things? LLMs in your Solutions Selected Scenarios Exciting Times… Democratizing Generative AI
  4. AI++: Integration von Large Language Models (LLMs) in moderne Business-Software

    Sprachverständnis ändert alles AI all-the-things? 4
  5. AI++: Integration von Large Language Models (LLMs) in moderne Business-Software

    Sprachverständnis ändert alles AI all-the-things? 6 Data Science Artificial Intelligence Machine Learning Unsupervised, supervised, reinforcement learning Deep Learning ANN, CNN, RNN etc. NLP Generative AI GAN, VAE, Transformers etc. Image / Video Generation GAN, VAE Large Language Models Transformers
  6. § LLMs generate text based on input § LLMs can

    understand text – this changes a lot § Without having to train them on data, domains, or use cases § Prompts are the universal interface (“UI”) → unstructured text with semantics § Human language evolves as a first-class citizen in software architecture 🤯 AI++: Integration von Large Language Models (LLMs) in moderne Business-Software Sprachverständnis ändert alles Large Language Models (LLMs) 7 Text… – really, just text?
  7. § LLMs are programs § LLMs are highly specialized neural

    networks § LLMs use(d) lots of data § LLMs need a lot of resources to be operated § LLMs have an API to be used through AI++: Integration von Large Language Models (LLMs) in moderne Business-Software Sprachverständnis ändert alles Large Language Models demystified 8
  8. AI++: Integration von Large Language Models (LLMs) in moderne Business-Software

    Sprachverständnis ändert alles LLMs in your Solutions 9
  9. AI++: Integration von Large Language Models (LLMs) in moderne Business-Software

    Sprachverständnis ändert alles Using LLMs: It’s just APIs ! Inference, FTW. 10
  10. GPT-4 API access OpenAI Playground AI++: Integration von Large Language

    Models (LLMs) in moderne Business-Software Sprachverständnis ändert alles DEMO 11
  11. AI++: Integration von Large Language Models (LLMs) in moderne Business-Software

    Sprachverständnis ändert alles The best tool for any developer to talk to LLMs! 12 🙈
  12. § OSS framework for developing applications powered by LLMs §

    > 1000 contributors § Python and Typescript versions § Chains for sequences of LLM-related actions in code § Abstractions for § Prompts & LLMs (local and remote) § Memory § Vector stores § Tools § Loading text from a wide range of sources AI++: Integration von Large Language Models (LLMs) in moderne Business-Software Sprachverständnis ändert alles LangChain - building LLM-based applications 13
  13. AI++: Integration von Large Language Models (LLMs) in moderne Business-Software

    Sprachverständnis ändert alles Talk to your Data 14
  14. AI++: Integration von Large Language Models (LLMs) in moderne Business-Software

    Sprachverständnis ändert alles Answering Questions on Data Retrieval-augmented generation (RAG) Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Text Question Answer LLM 15 Vector DB Embedding model Embedding model 💡 Indexing / Embedding QA
  15. RAG: Learning about my company’s policies via Slack LangChain (Python),

    Weaviate, Mixtral on Groq AI++: Integration von Large Language Models (LLMs) in moderne Business-Software Sprachverständnis ändert alles DEMO 16
  16. AI++: Integration von Large Language Models (LLMs) in moderne Business-Software

    Sprachverständnis ändert alles Talk to your Systems 17
  17. § LLM can be instructed to, e.g. § Do sentiment

    analysis § Extract information from text § Extract structured information § Schema description: JSON, TypeScript types, etc. § Clever & strict prompting § Tools like Kor, TypeChat, or Open AI Function / Tool Calling AI++: Integration von Large Language Models (LLMs) in moderne Business-Software Sprachverständnis ändert alles Extracting meaning in text for structured data 18
  18. Extracting structured data from text: Flight search OpenAI Tool Calling

    with LangChain, FastAPI & JS AI++: Integration von Large Language Models (LLMs) in moderne Business-Software Sprachverständnis ändert alles DEMO 19
  19. Extracting structured data from voice: Form filling JSON extraction &

    form filling in Angular, Mixtral on Groq AI++: Integration von Large Language Models (LLMs) in moderne Business-Software Sprachverständnis ändert alles DEMO 20
  20. § Tool / function calling standard established by OpenAI §

    Functions for interactions § LLM chooses to output JSON object containing arguments to call one or many functions § LLM does not call the function § You do it in your code § All major libs support tool calling with abstractions § OpenAI SDKs § Langchain § Semantic Kernel AI++: Integration von Large Language Models (LLMs) in moderne Business-Software Sprachverständnis ändert alles Extending LLM capabilities 21 curl https://api.openai.com/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-3.5-turbo", "messages": [ { "role": "user", "content": "What is the weather like in Boston?" } ], "tools": [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] } }, "required": ["location"] } } } ], "tool_choice": "auto" }'
  21. Extending LLM capabilities: Calculator as tool OpenAI Tool Calling with

    LangChain, Open AI GPT-3.5 AI++: Integration von Large Language Models (LLMs) in moderne Business-Software Sprachverständnis ändert alles DEMO 22
  22. AI++: Integration von Large Language Models (LLMs) in moderne Business-Software

    Sprachverständnis ändert alles Democratizing Generative AI 23
  23. AI++: Integration von Large Language Models (LLMs) in moderne Business-Software

    Sprachverständnis ändert alles LLMs everywhere OpenAI-related (cloud) OpenAI Azure OpenAI Service Big cloud providers Google Model Garden on Vertex AI Amazon Bedrock Open-source Edge IoT Server Desktop Mobile Web 24 Other providers Antrophic Cohere Mistral AI Hugging Face Open-source
  24. § Open-source community drives innovation in Generative AI § HuggingFace

    is central place for it § Literally, every week a new and “better” LLM shows up 🤓 § Important factors § Use case § Parameter size § Quantization § Processing power needed § Mistral-based family shows big potential for local use cases (7B params) AI++: Integration von Large Language Models (LLMs) in moderne Business-Software Sprachverständnis ändert alles Open-source LLMs thrive 25
  25. Local RAG with Mistral-7B open-source LLM llama.cpp, ollama, LangChain, StreamLit

    AI++: Integration von Large Language Models (LLMs) in moderne Business-Software Sprachverständnis ändert alles DEMO 26
  26. AI++: Integration von Large Language Models (LLMs) in moderne Business-Software

    Sprachverständnis ändert alles Exciting Times… 27
  27. § LLMs enable new scenarios & use cases to incorporate

    human language into software solutions § Fast moving and changing field § Every week something “big” happens in LLM space § Frameworks & ecosystem are evolving together with LLMs § Closed vs open LLMs § Competition drives invention & advancement § SLMs: specialized, fine-tuned for domains § Quality & security topics need to be addressed § SISO (sh*t in, sh*t out) § Quality of results heavily depends on your data & input AI++: Integration von Large Language Models (LLMs) in moderne Business-Software Sprachverständnis ändert alles Current state 28
  28. AI++: Integration von Large Language Models (LLMs) in moderne Business-Software

    Sprachverständnis ändert alles 2024: The year of SLMs 29