Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to use GPT with your Symfony application [EN]

How to use GPT with your Symfony application [EN]

With the release of ChatGPT, OpenAI has generated a lot of buzz around GPT and AI in general. For software architects and developers, this is not only a game changer when it comes to learning or debugging, but also introduces a new tool to embed into your system's architecture. In this talk, we will take a closer look at the basic concepts of GPT, the relevant models, and how you can effectively utilize them in your Symfony application.

Example Use Cases:
* https://github.com/chr-hertel/symfonycon-bot-2023
* https://github.com/chr-hertel/decomplex

Christopher Hertel

December 08, 2023
Tweet

More Decks by Christopher Hertel

Other Decks in Technology

Transcript

  1. Christopher Hertel - @el_stoffel What should I talk about? Alright,

    Christopher, that's easy: 1. LLM & GPT Start with the fundamentals of Large Language Models and GPT. 2. GPT Fundamentals Bring in fundamental concepts of GPT itself - best with a simple example. 3. Retrieval Augmented Generation Explain the basic concept of RAG and bring in a real-world example. 4. Function Calling Walk them through function calling - and if you have time, bring an example. 5. Q & A Well, let's see if they're still awake. SymfonyCon Brussels 2023
  2. Christopher Hertel - @el_stoffel LLM & GPT • Model •

    Result of training an AI algorithm with data, e.g. neural network • LLM == Large Language Models • Specific model to understand and generate language • Trained on a large amount of data, e.g. the internet • Basic idea is completion on top of probabilities • GPT == Generative Pre-Trained Transformer • First released by OpenAI • Generates language, word by word • Pre-trained on large data-sets • Transformer is a specific LLM architecture ChatGPT => ShopMVC
  3. Christopher Hertel - @el_stoffel Examples • Models • Open AI:

    gpt-3.5-turbo or gpt-4 • Meta: Llama 2 or CodeLlama • Anthropic: Claude • OSS: Falcon, GPT-J or GPT4All • BYO: Stanford Alpaca • Runtime • OpenAI & Microsoft Azure • huggingface.co • replicate.com • Self-hosted: GPT-J or GPT4All ChatGPT … • is not a model • is a product, using gpt-3.5 or gpt-4 • is for free for a reason
  4. Christopher Hertel - @el_stoffel Prompts • Messages that interact with

    the model • Style of prompts really important => “Prompt Engineering” • A lot of “trial & error” • Roles • User Messages created by a user • Assistant Messages created by a system • System High level instructions for the conversation
  5. Christopher Hertel - @el_stoffel You are ChatGPT, a large language

    model trained by OpenAI. You respond with only one sentence. System What should I do tomorrow? User That depends on your personal goals and priorities. Assistant
  6. Christopher Hertel - @el_stoffel What should I do tomorrow? User

    Do what makes you happy and fulfilled. Assistant You are my parent and always want "the best" for me. You respond only with one sentence. System
  7. Christopher Hertel - @el_stoffel What should I do tomorrow? User

    Get up and go for a run or try a new sport! Assistant You are a sports trainer that helps people to overcome their hesitation to do some sports. You respond with only one sentence. System
  8. Christopher Hertel - @el_stoffel What should I do tomorrow? User

    Make a to-do list and prioritize your tasks based on their importance and urgency. Assistant You are the assistant of Fabien Potencier, who is the creator of the PHP framework Symfony, and CPO of Platform.sh. You respond with only one sentence. System
  9. Christopher Hertel - @el_stoffel What should I do tomorrow? User

    Invest your time in something that will generate revenue. Assistant You are a capitalism fan boy and value money over people. You respond only with one sentence. System
  10. Christopher Hertel - @el_stoffel What should I do tomorrow? User

    Invest your time in something that will generate revenue. Assistant You are a capitalism fan boy and value money over people. You respond only with one sentence. System
  11. Christopher Hertel - @el_stoffel Context • Chat history => sum

    of messages • Provided all over again • The model interaction is stateless • state only training and fine-tuning • Limited by the number of “tokens” • gpt-3.5-turbo supports 16.385 tokens • gpt-4 supports 8.192 tokens • gpt-4-32k supports 32.768 tokens • gpt-4-turbo supports 128.000 tokens User Assistant System User Assistant User Assistant Context
  12. Christopher Hertel - @el_stoffel Temperature 0.0 2.0 creative deterministic •

    Mechanism how GPT handles probabilities • Often referred to as “randomness” More Options max tokens, n [of choices], penalty of presence or frequency, format, seed, top_p and tools
  13. Christopher Hertel - @el_stoffel Usage with Symfony • Architecture constraints

    • GPT is a component to understand and generate language • By default used via Web API • Libraries • HTTP clients like symfony/http-client or guzzlehttp/guzzle are sufficient • API client libraries like • openai-php/client with Symfony integration openai-php/symfony • orhanerday/open-ai that claims to be better • LLM Framework packages • Package kambo/langchain based on Langchain • Package theodo-group/llphant based on Langchain and LlamaIndex
  14. Christopher Hertel - @el_stoffel Use case: Implementation I • Install

    openai-php/symfony with Composer • Configure OpenAI token in environment variable $ composer require openai-php/symfony ###> openai-php/symfony ### # https://platform.openai.com/account/api-keys OPENAI_API_KEY=sk-rWYtxvD1dYouR34LlyTH1nkth1sI5real? OPENAI_ORGANIZATION= ###< openai-php/symfony ###
  15. Christopher Hertel - @el_stoffel Token • Sequences of characters found

    in texts Sequences of characters found in texts • Not necessarily same as words, especially around numbers or emojis Not necessarily same as words, especially around numbers or emojis • Foundation for GPT to understand semantics – every token has an ID Foundation for GPT to understand semantics Rule of Thumb words x 4/3 = tokens 75 words = 100 tokens
  16. Christopher Hertel - @el_stoffel Semantics negative positive professional jargon s**t

    insufficient cool excellent suboptimal lit boring bad fresh perfect
  17. Christopher Hertel - @el_stoffel Semantics negative positive professional jargon s**t

    insufficient cool excellent suboptimal lit boring bad fresh perfect High Semantic Similarity Low Semantic Similarity s**t insufficient cool excellent suboptimal lit boring bad fresh perfect
  18. Christopher Hertel - @el_stoffel Semantics negative positive professional jargon s**t

    insufficient cool excellent suboptimal lit boring bad fresh perfect High Semantic Similarity Low Semantic Similarity s**t insufficient cool excellent suboptimal lit boring bad fresh perfect +1 -1 +1 -1 [-1.0, -1.0] [-0.9, 0.8] [0.5, -0,4] [1.0, 1.0] [-0.1, 0.7] [0.8, -0.8] [-0.4, -0.4] [-0.5, 0.3] [0.3, -0.7] [0.6, 0.6]
  19. Christopher Hertel - @el_stoffel Vector • Used to express tokens,

    words, phrases and paragraphs as numerical data • Enables arithmetical calculations • Distance expresses semantic similarity • GPT uses 1536 dimensions per vector negative positive professional jargon s**t insufficient cool excellent suboptimal lit boring bad fresh
  20. Christopher Hertel - @el_stoffel Embeddings • Conversion/reduction of words &

    texts to (single) vector • The bigger the text, the “blurrier” the vector • Embedding models also provided by OpenAI, e.g. text-embedding-ada-002 Embeddings Model [0.547, -0.425, -0.152, …] [0.721, 0.111, -0.924, …] [-0.821, 0.289, -0.345, …] [0.424, 0.775, 0.522, …] [0.124, 0.542, -0.152, …]
  21. Christopher Hertel - @el_stoffel Retrieval Augmented Generation Data Source Vector

    Store 1 Create Embeddings Message Response 2 Create Embeddings 3 Similarity Search 4 Prompt & Generation Chain Chat Application
  22. Christopher Hertel - @el_stoffel Retrieval Augmented Generation Data Source Vector

    Store 1 Create Embeddings Message Response 2 Create Embeddings 3 Similarity Search 4 Prompt & Generation Chat Application
  23. Christopher Hertel - @el_stoffel Retrieval Augmented Generation Data Source Vector

    Store 1 Create Embeddings Message Response 2 Create Embeddings 3 Similarity Search 4 Prompt & Generation Chat Application TODO: - extended sketch
  24. Christopher Hertel - @el_stoffel Retrieval Augmented Generation • Combine GPT

    capabilities with domain-specific knowledge • Great to reduce hallucinations of GPT • Way more flexible than fine-tuning pre-trained models • Fine-grained control over data provided to the end user, e.g. user-specific • Easily combined with real time data
  25. Christopher Hertel - @el_stoffel Tools & Function Calling • Equipping

    GPT with functions • GPT delegates the call back to app • App comes back to GPT with results • GPT generates final response GPT App Chat UI Functions User Message Chain Response Prompts Model Model Response Function Function Call Result