Custom Chatbots with GPT and Symfony PHP framework

With the release of OpenAI's GPT, large language models gained a lot of awareness. Building custom chatbots is a great way to provide users with an interactive experience with your product or website. In this workshop, we will examine the fundamentals and build your very first GPT-based chatbot in our example domain.

Workshop at Web Summer Camp 2024:
https://websummercamp.com/2024/workshop/custom-chatbots-with-gpt-and-symfony-php-framework

Code Repository:
https://github.com/chr-hertel/wsc-symfony-chatbot

Transcript

1. CUSTOM CHATBOTS With GPT and Symfony PHP Framework Workshop by Christopher Hertel @ Web Summer Camp 2024
2. TODAY'S AGENDA 1. Basic Application Setup 2. Large Language Model 3. Prompts & Context 4. Vectors & Similarity 5. Retrieval Augmented Generation 6. Tools Mixing theory input & practical coding challenges.
3. WHO'S THAT GUY?
4. WORKSHOP APP
6. REQUIREMENTS What you need to follow this workshop: 1. Laptop 2. Internet Connection 3. Terminal & Browser 4. Git & GitHub Account 5. Docker with Docker Compose Plugin 6. Your Favorite IDE or Editor
7. TECHNOLOGY This small demo sits on top of following technologies: 1. PHP >= 8.3 2. Symfony 7.1 incl. Twig, Asset Mapper & UX 3. Bootstrap 5 4. OpenAI's GPT & Embeddings 5. ChromaDB Vector Store
8. BASIC APPLICATION SETUP Let's start with a git clone: See README for setup instructions. # ssh git clone git@github.com:chr-hertel/wsc-symfony-chatbot.git # https git clone https://github.com/chr-hertel/wsc-symfony-chatbot.git
9. SETTING UP OPENAI SECRETS 1. Go to bit.ly/wsc-chatbot 2. Copy file into config/secrets/dev/ 3. Run secrets command to test, see README
10. HELPER SCRIPT For convenience, we have a helper script: bin/check _ _ _____ _ _ _____ _ /\ | | | / ____| | | | | __ \ | | / \ | | | | | | |__ ___ ___| | _____ | |__) |_ _ ___ ___ ___ __| | / /\ \ | | | | | | '_ \ / _ \/ __| |/ / __| | ___/ _\ / __/ __|/ _ \/ _\ | / ____ \| | | | |____| | | | __/ (__| <\__ \ | | | (_| \__ \__ \ __/ (_| | /_/ \_\_|_| \_____|_| |_|\___|\___|_|\_\___/ |_| \__,_|___/___/\___|\__,_|
11. WHAT YOU SHOULD HAVE NOW 1. Symfony app running in your browser 2. Script bin/check reporting All Checks Passed 3. Project setup in your IDE or editor
12. LLM & GPT ChatGPT and beyond
13. LLM & GPT Model: Result of training an AI algorithm with data, e.g. neural network LLM == Large Language Models Model to understand and generate language Trained on a large amount of data, e.g. the www Basic idea is completion on top of probabilities
14. LLM & GPT GPT == Generative Pre-Trained Transformer First released by OpenAI Generates language, word by word Pre-trained on large data-sets Transformer is a specific LLM architecture
15. WHAT ARE WE USING? OpenAI's GPT-4o, released May 2024 GPT !== ChatGPT, product on top of GPT Available via OpenAI's API But there are other models: Meta's LLaMA, Google's Gemini, Anthropic's Claude, Mistral, Falcon, Stanford Alpaca, GPT-J, GPT4ALL, ...
16. CHALLENGE 1 Integrate basic GPT model with your chatbot
17. BASIC GPT USAGE Merge branch 1-gpt into your working branch In the end bin/check should be green again. Implement App\OpenAI\GptClient Use App\OpenAI\GptClientInterface in App\Chat See CHALLENGES.md for guidance git fetch origin git merge origin/1-gpt bin/check # will fail
18. EXAMPLE API CALL curl https://api.openai.com/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-4o", "temperature": 1.0, "messages": [ {"role": "user", "content": "Hello!"} ] }'
19. ARE YOU READY? bin/check _ _ _____ _ _ _____ _ /\ | | | / ____| | | | | __ \ | | / \ | | | | | | |__ ___ ___| | _____ | |__) |_ _ ___ ___ ___ __| | / /\ \ | | | | | | '_ \ / _ \/ __| |/ / __| | ___/ _\ / __/ __|/ _ \/ _\ | / ____ \| | | | |____| | | | __/ (__| <\__ \ | | | (_| \__ \__ \ __/ (_| | /_/ \_\_|_| \_____|_| |_|\___|\___|_|\_\___/ |_| \__,_|___/___/\___|\__,_|
20. ARE YOU READY?
21. GPT FUNDAMENTALS Prompts, Temperature, and Context
22. PROMPTS Messages that interact with the model Style of prompts really important => Prompt Engineering A lot of "trial & error" Roles User: Messages created by a user Assistant: Messages created by a system System:Instructions for the conversation
23. TEMPERATURE Mechanism how GPT handles probabilities Often referred to as "randomness" High temperature: more creative Low temperature: more deterministic
24. CONTEXT Collection of prompts & messages Basis for the model to generate new text Powerful to control the model
25. EXAMPLE
28. CHALLENGE 2 Extend the chat's context to bring in more knowledge
29. EXTENDED CONTEXT Merge branch 2-context into your working branch In the end bin/check should be green again. Inject knowledge about date, time and Web Summer Camp program into context. Implement two decorators for GptClient See CHALLENGES.md for more guidance git fetch origin git merge origin/2-context bin/check # will fail
30. ARE YOU READY? bin/check _ _ _____ _ _ _____ _ /\ | | | / ____| | | | | __ \ | | / \ | | | | | | |__ ___ ___| | _____ | |__) |_ _ ___ ___ ___ __| | / /\ \ | | | | | | '_ \ / _ \/ __| |/ / __| | ___/ _\ / __/ __|/ _ \/ _\ | / ____ \| | | | |____| | | | __/ (__| <\__ \ | | | (_| \__ \__ \ __/ (_| | /_/ \_\_|_| \_____|_| |_|\___|\___|_|\_\___/ |_| \__,_|___/___/\___|\__,_|
31. ARE YOU READY?
33. RETRIEVAL AUGMENTED GENERATION Semantic Vectors & Similarity Search
34. TOKEN Sequences of characters found in texts Not necessarily same as words Foundation for GPT to understand semantics
35. TOKEN Sequences of characters found in texts Not necessarily same as words Foundation for GPT to understand semantics
36. SEMANTICS
37. SEMANTICS
38. SEMANTICS
39. VECTOR Expresses tokens as numerical data Enables arithmetical operations Distance == similarity GPT uses 1536 dimensions
40. EMBEDDINGS Convert words/texts into single vector The bigger the text, the "blurrier" the vector OpenAI also provides embedding models e.g. text-embedding-ada-002
41. EMBEDDINGS
42. RETRIEVAL AUGMENTED GENERATION Combines GPT with embeddings Embeddings used for similarity search Allows to inject knowledge into GPT
44. CHALLENGE 3 Convert Program into Embeddings in ChromaDB
45. TURN PROGRAM INTO EMBEDDINGS Merge branch 3-vectors into your working branch Test with app:test:chroma and app:program:embed Implement App\OpenAI\EmbeddingClient for API Implement App\WscProgram\Embedder See CHALLENGES.md for more guidance git fetch origin git merge origin/3-vectors bin/check # will fail
46. ARE YOU READY? docker compose exec app bin/console app:test:chroma Testing Chroma DB Connection ============================ // Connecting to Chroma DB ... ------------------ -------------------------------------- Key Value ------------------ -------------------------------------- ChromaDB Version 0.5.3 Collection Name wsc-program Collection ID 18bacf09-3512-40bd-8077-895c6e1c98ff Total Documents 34 ------------------ -------------------------------------- // Searching for Symfony content ...
47. CHALLENGE 4 Switch to RAG instead of static context
48. RETRIEVAL AUGMENTED GENERATION Merge branch 4-retrieval into your working branch git fetch origin git merge origin/4-retrieval bin/check # will fail
49. DECORATORS Use Symfony feature to implement decorators #[AsDecorator(decorates: GptClientInterface::class, priority: 10)] final class RetrievalClient implements GptClientInterface
50. ARE YOU READY? bin/check _ _ _____ _ _ _____ _ /\ | | | / ____| | | | | __ \ | | / \ | | | | | | |__ ___ ___| | _____ | |__) |_ _ ___ ___ ___ __| | / /\ \ | | | | | | '_ \ / _ \/ __| |/ / __| | ___/ _\ / __/ __|/ _ \/ _\ | / ____ \| | | | |____| | | | __/ (__| <\__ \ | | | (_| \__ \__ \ __/ (_| | /_/ \_\_|_| \_____|_| |_|\___|\___|_|\_\___/ |_| \__,_|___/___/\___|\__,_|
51. TOOLS Equipping GPT with more Power
52. TOOLS GPT delegates calls back to app App comes back to GPT with result GPT generates final respoonse
55. API PAYLOAD "tools": [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and country, eg. San Francisco, USA" }, "format": { "type": "string", "enum": ["celsius", "fahrenheit"] } }
56. LLM CHAIN Easier to use ToolChain of php-llm/llm-chain use PhpLlm\LlmChain\ToolBox\AsTool; #[AsTool('clock', 'Provides the current date and time', '__invoke')] final class Clock
57. CHALLENGE 5 Shift time and program search to tools
58. TIME & PROGRAM AS TOOLS Merge branch 5-tools into your working branch ToolChain instead of GptClientInterface. Create App\Tool\Clock & register AsTool Create App\Tool\Retriever & register AsTool git fetch origin git merge origin/5-tools bin/check # will fail
59. ARE YOU READY? bin/check _ _ _____ _ _ _____ _ /\ | | | / ____| | | | | __ \ | | / \ | | | | | | |__ ___ ___| | _____ | |__) |_ _ ___ ___ ___ __| | / /\ \ | | | | | | '_ \ / _ \/ __| |/ / __| | ___/ _\ / __/ __|/ _ \/ _\ | / ____ \| | | | |____| | | | __/ (__| <\__ \ | | | (_| \__ \__ \ __/ (_| | /_/ \_\_|_| \_____|_| |_|\___|\___|_|\_\___/ |_| \__,_|___/___/\___|\__,_|
60. CHALLENGE 6 Chat on top of YouTube Transcript
61. YOUTUBE TRANSCRIPT BOT Merge branch 6-youtube into your working branch New bot with YouTubeComponent Implement App\YouTube Use App\YouTube\TranscriptFetcher git fetch origin git merge origin/6-youtube
62. ARE YOU READY? bin/check _ _ _____ _ _ _____ _ /\ | | | / ____| | | | | __ \ | | / \ | | | | | | |__ ___ ___| | _____ | |__) |_ _ ___ ___ ___ __| | / /\ \ | | | | | | '_ \ / _ \/ __| |/ / __| | ___/ _\ / __/ __|/ _ \/ _\ | / ____ \| | | | |____| | | | __/ (__| <\__ \ | | | (_| \__ \__ \ __/ (_| | /_/ \_\_|_| \_____|_| |_|\___|\___|_|\_\___/ |_| \__,_|___/___/\___|\__,_|
63. CHALLENGE 7 Tool Chain on top of Wikipedia
64. WIKIPEDIA TOOL CHAIN Merge branch 7-wikipedia into your working branch New Wikipedia bot for research Implement App\Wikipedia Use App\Wikipedia\Client git fetch origin git merge origin/7-wikipedia
65. ARE YOU READY? bin/check _ _ _____ _ _ _____ _ /\ | | | / ____| | | | | __ \ | | / \ | | | | | | |__ ___ ___| | _____ | |__) |_ _ ___ ___ ___ __| | / /\ \ | | | | | | '_ \ / _ \/ __| |/ / __| | ___/ _\ / __/ __|/ _ \/ _\ | / ____ \| | | | |____| | | | __/ (__| <\__ \ | | | (_| \__ \__ \ __/ (_| | /_/ \_\_|_| \_____|_| |_|\___|\___|_|\_\___/ |_| \__,_|___/___/\___|\__,_|
66. THANKS FOR JOINING! I'll be around for feedback, questions & discussions. Get in contact via christopher-hertel.de