LLM Development Landscape

LLM Development Landscape
Presented at Data + AI Day 2024
6th October 2024

Transcript

1. LLM Development Landscape Kamolphan Liwprasert (Fon) MLOps Consultant, AIMET.tech Google Developer Expert - Cloud
2. แนะนําตัว
3. AIMET aimet.tech
4. LLM Development Landscape ✨ Overview ภาพรวมในการพัฒนาแอป LLM ✨ Concept ที่น่ารู้เกี่ยวกับ LLM ✨ Dev Application LLM อย่างไรได้บ้าง ✨ มี framework อะไรให้เลือกใช้บ้าง
8. Overview ภาพรวม การพัฒนาแอป LLM
9. นิยาม LLM A large language model (LLM) is a computational model capable of language generation or other natural language processing tasks. https://en.wikipedia.org/wiki/Large_language_model
10. นิยาม Multimodal LLM Multimodal = characterized by several different modes of activity or occurrence. https://research.google/blog/multimodal-medical-ai/
11. เราเรียกใช้ AI Model อย่างไรได้บ้าง?
12. Model Serving Application 📱 💻 🌐 Model 🤖 API API = “Client - Server” Client Server
13. Application 📱 💻 🌐 Model On-Device AI = “Edge”
14. Challenge?
16. Language Model APIs
17. 🏆 LMSYS Chatbot Arena Leaderboard https://chat.lmsys.org/?leaderboard
18. Artificial Analysis: เว็บเปรียบเทียบ AI models https://artificialanalysis.ai/
19. Artificial Analysis: Quality vs Price https://artificialanalysis.ai/
20. Artificial Analysis: API Prices https://artificialanalysis.ai/
21. Services to Host Language Models
22. Why self-host LLM? 💲 Cost efficient in long term (ie. on-premise) → Need to tune the latency to make the model faster ⚙ Customization & fine-tuning → No lock-in to a particular model 🔒 Security compliance & data residency / privacy
23. Run LLM locally LlamaFile github.com/Mozilla-Ocho/llamafile Ollama ollama.com/ LM Studio lmstudio.ai/
24. LLM Development Frameworks
25. LangChain 🦜🔗 Python / JS library framework for developing applications powered by large language models (LLMs). https://www.langchain.com/langchain
26. LlamaIndex Turn your enterprise data into production-ready LLM applications. (Python / Typescript) https://www.llamaindex.ai/
27. Semantic Kernel from Microsoft Semantic Kernel is an SDK that integrates Large Language Models (LLMs) like OpenAI, Azure OpenAI, and Hugging Face with conventional programming languages like C#, Python, and Java. https://github.com/microsoft/semantic-kernel
28. It’s fine not using any of these frameworks 󰙤
29. RAG Concept :Retrieval Augmented Generator
30. RAG - Ask→ Retrieve from DB → Generate Answer
31. Document Search example Vector DB
32. Vector Database https://www.graft.com/blog/top-vector-databases-for-ai-projects
33. RAG vs Fine-tuning
34. Agentic Workflow Agentic = behaves like an agent
35. Why Agentic? https://www.vellum.ai/blog/agentic-workflows-emerging-architectures-and-design-patterns
36. Agentic Workflow https://www.vellum.ai/blog/agentic-workflows-emerging-architectures-and-design-patterns
37. Crew AI https://www.crewai.com/
38. https://github.com/microsoft/autogen
39. Azure: Copilot Studio GCP: VertexAI Agent Builder
40. Inference / Serving
41. Text Generation Inference https://huggingface.co/docs/text-generation-inference/index
42. vLLM = Model serving for LLM Easy, fast, and cheap LLM serving for everyone vLLM is fast with: ✅ State-of-the-art serving throughput ✅ Efficient management of attention key and value memory with PagedAttention ✅ Continuous batching of incoming requests ✅ Fast model execution with CUDA/HIP graph ✅ Quantization: GPTQ, AWQ, SqueezeLLM, FP8 KV Cache ✅ Optimized CUDA kernels https://github.com/vllm-project/vllm Throughput: Higher is better
43. Responsible AI
44. https://ai.google/responsibility/responsible-ai-practices/
45. Google's Secure AI Framework https://safety.google/cybersecurity-advancements/saif/
46. Responsible AI ✅ ตรวจสอบความถูกต้องเสมอ ✅ Human-centered Design ออกแบบสําหรับคนใช้ ⚠ ระวังเรื่อง Data Privacy ความเป็นส่วนตัวของข้อมูล ⚠ Biases and Fairness ทําให้มีความเป็นธรรมกับผู้ใช้
47. Resources
48. https://www.promptingguide.ai/
49. Sunday 3 November 2024 @ K+ Building Samyan Register now: bit.ly/devfest-cloud-bkk24 Saturday 26 October 2024 @ Cleverse Register now: bit.ly/technologista-2024 ฝาก event :) Technologista By PyLadies x Women Techmakers DevFest Cloud Bangkok By GDG Cloud Bangkok
50. LLM Development Landscape Kamolphan Liwprasert (Fon) MLOps Consultant, AIMET.tech Google Developer Expert - Cloud