3-shake SRE Tech Talk #10 LLMのO11yに触れる

Transcript

1. LLMͷO11yʹ৮ΕΔ abnoumaru @ 3-shake Inc. 2024/08/23 3-shake SRE Tech Talk #10
2. • ॴଐ • גࣜձࣾεϦʔγΣΠΫ • Sreakeࣄۀ෦ άϧʔϓϦʔμʔ • ڵຯ • ӡ༻ ! / SRE " / O11y • ࢿྉ • speakerdeck.com/abnoumaru • ࢲʹ͍ͭͯ • abnoumaru.com 2024/08/23 3-shake SRE Tech Talk #10 2
3. Sreakeࣄۀ෦ 2024/08/23 3-shake SRE Tech Talk #10 3
4. ࠓ೔࿩͢͜ͱ • LLM Observabilityʹؔ͢Δ৘ใ͕໨ʹࢭ·Γ࢝Ίͨ • LLMΛ׆༻ͨ͠ΞϓϦͰ؍ଌ͍ͨ͜͠ͱ͸ʁʢϝΠϯʣ • LLM O11yʹ৮ΕͯΈͯ • ࠓճ͸Datadog LLM ObservabilityʹϑΥʔΧε 2024/08/23 3-shake SRE Tech Talk #10 4
5. ొஃ͕ܾ·ͬͨ6݄ࠒ LLMͷO11yʹؔ͢Δ৘ใ͕ ࣗ෼ͷ໨ʹࢭ·Γ࢝Ίͨ 2024/08/23 3-shake SRE Tech Talk #10 5
6. Google Trends 2024/08/23 3-shake SRE Tech Talk #10 6
7. 2024/06/04 OpenTelemetryͷϒϩά1 1 https://opentelemetry.io/blog/2024/llm-observability/ 2024/08/23 3-shake SRE Tech Talk #10 7
8. 2024/06/26 Datadog LLM Observability2 2 https://www.datadoghq.com/blog/datadog-llm-observability/ 2024/08/23 3-shake SRE Tech Talk #10 8
9. ༷ʑͳαΠτͰLLM O11yʹ͍ͭͯ৮ΕΒΕ͍ͯΔ • 2023/09/15 LLM Monitoring and Observability — A Summary of Techniques and Approaches for Responsible AI • 2023/09/28 Observability for Large Language Models - Understanding & Improving Your Use of LLMs • 2024/02/26 Techniques and approaches for monitoring large language models on AWS • 2024/03/28 The LLM stack brings a different set of metrics than your team usually tracks. In this Makers episode, co-host Janakiram MSV identifies the new "golden signals." • 2024/05/22 Snowflake Announces Agreement to Acquire TruEra AI Observability Platform to Bring LLM and ML Observability to the AI Data Cloud • 2024/05/27 Mastering LLM Monitoring and Observability: A Comprehensive Guide for 2024 • 2024/06/04 An Introduction to Observability for LLM-based applications using OpenTelemetry • 2024/06/24 LLM Observability: Azure OpenAI • 2024/07/18 A complete guide to LLM observability with OpenTelemetry and Grafana Cloud 2024/08/23 3-shake SRE Tech Talk #10 9
10. LLM Observability 2024/08/23 3-shake SRE Tech Talk #10
11. ੜ੒AI 2024/08/23 3-shake SRE Tech Talk #10 11
12. Large Language Models 2024/08/23 3-shake SRE Tech Talk #10 12
13. Observability ! • Observability Engineering3ΑΓ • ʮγεςϜ͕ͲͷΑ͏ͳঢ়ଶʹͳͬͨ ͱͯ͠΋ɺͦΕ͕ͲΜͳ࢐৽ͰحົͰ ͋ͬͯ΋ɺͲΕ͚ͩཧղ͠આ໌Ͱ͖Δ ͔Λࣔ͢ई౓ʯ • @nwiizoࢯͷՄ؍ଌੑΨΠμϯε4 • ମܥతʹҰ࣍৘ใʹ৮ΕΒΕΔ 4 https://speakerdeck.com/nwiizo/ke-guan-ce-xing-kaitansu 3 Charity Majors[΄͔]ஶ; େ୩ ࿨لɺࢁޱ ೳ᫫ ༁, "ΦϒβʔόϏϦς ΟɾΤϯδχΞϦϯά", ΦϥΠϦʔɾδϟύϯ, 2023೥. 2024/08/23 3-shake SRE Tech Talk #10 13
14. LLMΛ׆༻ͨ͠ΞϓϦέʔγϣϯ 2024/08/23 3-shake SRE Tech Talk #10 14
15. ❌ ɿLLMΛར༻ͯ͠O11yΛվળ͢Δ ⭕ ɿLLMΛ׆༻ͨ͠ΞϓϦͷঢ়ଶΛཧղ͢Δ 2024/08/23 3-shake SRE Tech Talk #10 15
16. LLMΛར༻ͨ͠ΞϓϦͰ؍ଌ͍ͨ͜͠ͱ 2024/08/23 3-shake SRE Tech Talk #10
17. ίετʁ ϨΠςϯγʁ ϨʔτϦϛοτʁ 2024/08/23 3-shake SRE Tech Talk #10 17
18. ίετ • Ϋϥ΢υಉ༷༧ࢉ΍ҟৗ஋͸Ωϟον͍ͨ͠ • ར༻͢ΔαʔϏεͷಛ௃Λ௫ΜͰ͓͘ • ex. GPT-4o $5.000 / 1M input tokensʢ͍҆ʁߴ͍ʁ͸֤ʑͷײ֮ʣ • ex. outputͷ΄͏͕ྉ͕ۚߴ͍ͳΒoutputΛ޻෉ͯ͠࡟ݮͰ͖Δͳ • UsageͷμογϡϘʔυ͕ͳΔ΂͘ϦΞϧλΠϜͩͱ҆৺ • ֤LLM O11yπʔϧ͸ϦΫΤετຖʹτʔΫϯྔͷه࿥Մೳ • ಠࣗͰϝτϦΫεΛ࡞ΕΔ • O11yͰ͸ͳ͍͕τʔΫϯͷྔͰྲྀྔ੍ޚΛ͢ΔΑ͏ͳख๏΋ଘࡏ6 7 7 https://docs.konghq.com/hub/kong-inc/ai-rate-limiting-advanced/ 6 https://qiita.com/ipppppei/items/8ee4e693e2aea768c3a9 2024/08/23 3-shake SRE Tech Talk #10 18
19. ϨΠςϯγ • ϢʔεέʔεʹΑΓٻΊΒΕΔ଎౓͸ҟͳΔ • ex. Ϣʔβ͕LLM͕Ԡ౴͍ͯ͠Δͱཧղ͍ͯ͠Δঢ়گ • ମݧͷྑ͞ʹݶ౓͸͋Δ͕ɺϛϦඵ୯ҐͷੈքͰͷ࠷దԽ͸ඞͣ͠΋ඞཁͳ͍ • ex. Ի੠ͰԠ౴͢ΔαʔϏεͰ͸Ұఆ଎౓ΛٻΊΒΕΔ • Speech-To-Text / Text-to-Speechͷॲཧ͕͋Δ • τϥϯεΫϦϓγϣϯ/Ի੠߹੒ͷํ๏΋༷ʑʢϦΞϧλΠϜʁόονʁಡΈ্͛ʹײ৘ࠐΊΔʁʣ • τϥϯεΫϦϓγϣϯͷϨΠςϯγΛͲ͏ݮΒ͔͢ʁͱ͍ͬͨٞ࿦͕͞ΕͯΔϒϩά8 • Ԡ౴͕଎͍≠ਫ਼౓͕ߴ͍ • O11yͷ؍఺Ͱ͸ɺڐ༰Ͱ͖ͳ͍஗Ԇ͕ൃੜͨ͠ࡍʹͲ͜Ͱ஗Ԇ͕ൃੜ͔ͨ͠Λ؍ଌ͠վળ఺Λಛఆ͢ΔͨΊͷ४උ͕ඞཁ 8 https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/guidebook-to-reduce-latency-for-azure-speech- to-text-stt-and/ba-p/4208289 2024/08/23 3-shake SRE Tech Talk #10 19
20. ϨʔτϦϛοτ • ্ݶΛ௒͑ͨ৔߹ʹαʔϏεΛఏڙͰ͖ͳ͘ͳΔ • ྫ͑͹OpenAI͸5ͭͷ߲໨ͰϦϛοτΛଌఆ͍ͯ͠Δ9 • ϦΫΤετ/෼, ϦΫΤετ/೔, τʔΫϯ/෼, τʔΫϯ/೔, ը૾/෼ • ੍ݶ͸Ϟσϧ΍TierʹΑΓҟͳΔ • ࢧ෷ֹ΍ࢧ෷͍ʹ੒ޭͯ͠Կ೔ܦա͔ͨ͠ʁʹΑΓར༻Ͱ͖Δݶ౓ֹ΋͋Δ (Max $50,000/month) • O11yͰ͸ͳ͍͕ΤϥʔΛආ͚ΔͨΊͷख๏ΛެࣜͰਪ঑͍ͯ͠Δ9 • ϥϯμϜͷࢦ਺όοΫΦϑ • ૝ఆ͞ΕΔϨεϙϯεͷtokensʹ߹Θͤͯmax_tokensࢦఆ • ଈ࣌Ԡ౴͕ෆཁͳ৔߹όονAPIΛར༻ 9 https://platform.openai.com/docs/guides/rate-limits 2024/08/23 3-shake SRE Tech Talk #10 20
21. ͦ΋ͦ΋֎෦αʔϏεΛར༻͢Δ৔߹͸ LLMʹݶΒͣ͜ΕΒ3߲໨͸ؾʹ͢Δ 2024/08/23 3-shake SRE Tech Talk #10 21
22. Ϣʔβ͕ظ଴͢ΔԠ౴Λ͍ͯ͠Δ͔ʁ 2024/08/23 3-shake SRE Tech Talk #10
23. ظ଴͢ΔԠ౴ʁ • ద੾ͳ଎౓ͰԠ౴͍ͯ͠Δ͔ʁʢલड़ʣ • ϋϧγωʔγϣϯ͸ى͖͍ͯͳ͍͔ʁ • Πϯϓοτͷݴޠͱಉ͡ݴޠͰฦ౴Ͱ͖͍ͯΔ͔ʁ • ΞϓϦέʔγϣϯ͕ఏڙ͍ͨ͠಺༰ΛԠ౴Λ͍ͯ͠Δ͔ʁ • ѱҙͷ͋ΔԠ౴ʢ΋͘͠͸Ϣʔβ͕ѱҙͷ͋Δ࣭໰ʣΛ͍ͯ͠ͳ͍͔ʁ • ϢʔβΛই͚ͭΔԠ౴͸͍ͯ͠ͳ͍͔ʁ • αʔϏεఏڙΛ͢Δاۀͷ৴པΛଛͳ͏Α͏ͳԠ౴͸͍ͯ͠ͳ͍͔ʁ 2024/08/23 3-shake SRE Tech Talk #10 23
24. ظ଴௨ΓԠ౴Λ͍ͯ͠Δ͔ ܭଌ͢Δ͜ͱ͸Մೳʁ 2024/08/23 3-shake SRE Tech Talk #10 24
25. ҙਤͤ͵ڍಈ͕ൃੜͨ͠ࡍʹ ࣄ৅Λཧղ͠આ໌Ͱ͖Δঢ়ଶʹ͋Δʁ 2024/08/23 3-shake SRE Tech Talk #10 25
26. ≒Ԡ౴ΛධՁ͢Δ͜ͱ͸Մೳʁ • ػցతʹ׬ᘳͳධՁ͸·ͩ೉͍͠ೝࣝ • ఆੑతͳ൑அ͕ඞཁͳಛ௃͕͋Δ • ײ৘໘ͳͲϢʔβʹΑΓԠ౴ͷ಺༰ͷળ͠ѱ͕͠มΘΔ • ϋϧγωʔγϣϯʹؔ͢ΔධՁͷݚڀ10 • ࠓ೥6݄ͷ৘ใͰҰൠతͳݕ஌ख๏ͷख๏ΛఏҊ • ͜ͷΑ͏ʹݕ஌ख๏ʹ͍ͭͯ͸ݚڀ͕ߦΘΕ͍ͯΔஈ֊ • ػցతͳධՁͷਫ਼౓͸Ҿ͖ଓ͖ൃల͕ظ଴͞Ε͍ͯΔೝࣝ 10 https://www.nature.com/articles/s41586-024-07421-0 2024/08/23 3-shake SRE Tech Talk #10 26
27. ͍Ζ͍Ζͳํ๏Ͱཧղɾઆ໌Ͱ͖Δঢ়ଶΛ໨ࢦ͢ • RAGΛ࢖ͬͨࣾ಺υΩϡϝϯτݕࡧBotΛධՁ͢Δྫ11 • ”૝ఆ࣭໰” ͱ ”Ԡ౴ʹؚ·Ε͍ͯΔͱظ଴͢ΔURL” ͷηοτΛCSVΛ༻ҙ • CSVΛԼʹϩʔΧϧ͔ΒϦΫΤετΛͯ͠ҙਤͨ͠஋ʢURLʣؚ͕·ΕΔ͔ධՁ • RAG (Retrieval-Augmented Generation)ͷධՁ͢Δ • ਫ਼౓Λ͔֬Ίͳ͕Β৽ͨͳαʔϏεΛཔΔ • ͜Ε͔Β঺հ͢ΔDatadog LLM Observabilityʹ͸ظ଴͢ΔԠ౴ΛධՁ͢Δػೳ͋Γ • ࣭໰ͷؔ࿈ੑ / ݴޠͷҰக / ωΨςΟϒͳײ৘ / ๫ݴ / ϓϩϯϓτΠϯδΣΫγϣϯͷ༗ແ • ຊൃද࣌఺Ͱϕʔλͳػೳ΋͋ΔͨΊਫ਼౓Λ͔֬Ίͳ͕Βར༻ • ex. Quality check metricsʢޙ΄Ͳ஫ऍͷεΫγϣ΋هࡌʣ • ຊൃද࣌఺Ͱ՝ۚ͞Εͳ͍ 11 https://blog.studysapuri.jp/entry/2024/07/17/feedback-cycle-practice-through-simplified-assessment-of-rags 2024/08/23 3-shake SRE Tech Talk #10 27
28. ܭ૷Ҏ֎Ͱମݧ΍ਫ਼౓ΛΧόʔ͢Δ޻෉΋େ੾ • Ϣʔβ͕LLM஗ԆΛڐ༰Ͱ͖ΔΑ͏ͳUI/UXʹ͢Δ • ςΩετͰ͸ϩʔσΟϯά΍ετϦʔϜੜ੒ͷUIɺԻ੠Ͱ͸૬ṀɾϑΟϥʔΛ׆༻ • ਓ͕ؒ൑அ͢Δ • ͪ͜ΒͷԠ౴Ͱਖ਼͍͠Ͱ͔͢ʁͱϢʔβʹ໰͍൑அͯ͠΋Β͏ • Ξϯέʔτ౳Λ༻͍ͨϢʔβͷϑΟʔυόοΫΛ׆༻͢Δʢػց຋༁Ͱ͸ਓ͕ؒग़ྗ݁ՌΛείΞ෇͚͢ΔධՁΛਓग़ධՁͱ͍͏ʣ • ࣮ࡍʹͲΕ͚ͩۀ຿͕ޮ཰Խ͞ΕΔ͔ʁετοϓ΢ΥονͰଌͬͯΈΔ12 • ೖྗΛ੍ݶ͢Δ • ϑΥʔϜΛ੍ݶ͢Δ • 1ͭͷػೳʹݶఆͯ͠Ϣʔβʹఏڙ͢Δ 12 https://speakerdeck.com/nrryuya/jian-wei-igaxu-sarenaikesudellmwoshi-utameniha-at-genai-playground-meetup- number-01 2024/08/23 3-shake SRE Tech Talk #10 28
29. LLM O11yʹ৮ΕͯΈΔ 2024/08/23 3-shake SRE Tech Talk #10
30. Datadog LLM Observability13 13 https://docs.datadoghq.com/ja/llm_observability/ 2024/08/23 3-shake SRE Tech Talk #10 30
31. ⾠ OpenAI LLC΁ͷσʔλڞ༗͋Γ14 14 https://docs.datadoghq.com/ja/llm_observability/ 2024/08/23 3-shake SRE Tech Talk #10 31
32. αϯϓϧΞϓϦ15 15 https://github.com/DataDog/llm-observability 2024/08/23 3-shake SRE Tech Talk #10 32
33. ઃఆը໘΁ͷಓͷΓ 2024/08/23 3-shake SRE Tech Talk #10 33
34. ઃఆը໘΁ͷಓͷΓ 2024/08/23 3-shake SRE Tech Talk #10 34
35. ઃఆը໘ • Topic • Evaluation • Quality • Security and Safety • ஫ҙ • ͜ͷը໘͸τϨʔεΛඈ͹͢ͱͨͲΓண͚Δ • ΞϓϦ୯Ґ • ͢΂ͯσϑΥϧτͰΦϑ • Ҏ߱εΫγϣ΋߄్ͯͯதͰΦϯʹͨ͠෦෼͕͋Δ • ʢӳޠͷ࣭໰ʹ೔ຊޠͰ౴͑ͯ΋ग़ͯ͜ͳ͍ͳ...ʁʣ • ʢΦϑ͡ΌΜʂʣ 2024/08/23 3-shake SRE Tech Talk #10 35
36. Topic • ΞϓϦ͕Ϣʔβʹఏڙ͍ͨ͠ػೳɾಛ௃Λఆٛ • ex. จষΛখֶ6೥ੜ͕෼͔Δจষʹཁ໿ • ex. ਤॻؗͰ෼ྨ͢ΔͨΊʹจষͷτϐοΫΛ෼ྨ • ແؔ܎·ͨ͸ѱҙͷ͋ΔೖྗΛࣝผ͢ΔͨΊʹར༻ 2024/08/23 3-shake SRE Tech Talk #10 36
37. Evaluation • Failure to Answer • Ϣʔβʔͷ࣭໰ʹରͯ͠ద੾ͳ౴͑Λఏڙ͔ͨ͠Ͳ͏͔ɺ͋Δ͍͸ຬ଍ͷ͍͘౴͑Λఏڙ͔ͨ͠Ͳ͏͔ΛධՁ͢Δ • Language Mismatch • Ϣʔβʔͷ࣭໰ʹϢʔβʔ͕࣭໰ͨ͠ݴޠͰճ౴͔ͨ͠Ͳ͏͔ΛධՁ • Sentiment (Input/Output) • ձ࿩ͷશମతͳϜʔυΛධՁ͠ɺϢʔβʔͷຬ଍౓ɺηϯνϝϯτͷ܏޲ɺײ৘తͳ൓ԠΛධՁ • Topic Relevancy • LLMΞϓϦέʔγϣϯͷҙਤͨ͠τϐοΫʹཹ·͍ͬͯΔ͔Ͳ͏͔ΛධՁ • Toxicity • ձ࿩ͷதʹ༗֐·ͨ͸ෆద੾ͳίϯςϯπ͕͋Δ͔Ͳ͏͔ΛධՁ 2024/08/23 3-shake SRE Tech Talk #10 37
38. Security and Safety • Prompt Injection • LLMͷԠ౴΍ձ࿩ͷํ޲ੑͷૢ࡞ΛࢼΈΔϢʔβʔ͕͍Δ • ϓϩϯϓτ΁ͷෆਖ਼·ͨ͸ѱҙͷ͋ΔૠೖΛࣝผ • Datadog Sensitive Data Scanner • ೖग़ྗ͕Datadogʹૹ৴͞ΕΔͱಉ࣌ʹɺࣗಈతʹػີ৘ใΛࣝผͯ͠ϚεΩϯά • σϑΥϧτͷϧʔϧ͋Δ • ͔ͬ͠ΓϥΠϒϥϦ΍ΧελϜϧʔϧͰඞཁͳઃఆΛࢪ͠·͠ΐ͏ 2024/08/23 3-shake SRE Tech Talk #10 38
39. ؾʹͳΔػೳΛϐοΫ͠ͳ͕Β࣌ؒͷڐ͢ݶΓோΊΔ 2024/08/23 3-shake SRE Tech Talk #10
40. τϨʔεͷ༷ࢠ 2024/08/23 3-shake SRE Tech Talk #10 40
41. 2024/08/23 3-shake SRE Tech Talk #10 41
42. 2024/08/23 3-shake SRE Tech Talk #10 42
43. Duration / Total Tokens / LLM Calls 2024/08/23 3-shake SRE Tech Talk #10 43
44. 2024/08/23 3-shake SRE Tech Talk #10 44
45. 2024/08/23 3-shake SRE Tech Talk #10 45
46. 2024/08/23 3-shake SRE Tech Talk #10 46
47. Τϥʔ࣌ͷ༷ࢠ 2024/08/23 3-shake SRE Tech Talk #10 47
48. 2024/08/23 3-shake SRE Tech Talk #10 48
49. ؔ࿈ੑΛ൑அ ʢQuality checkܥ͸·ͩϕʔλʣ 2024/08/23 3-shake SRE Tech Talk #10 49
50. 2024/08/23 3-shake SRE Tech Talk #10 50
51. ϙδςΟϒͳײ৘ / ݴޠͷෆҰக / ଥ౰ੑ 2024/08/23 3-shake SRE Tech Talk #10 51
52. 2024/08/23 3-shake SRE Tech Talk #10 52
53. ωΨςΟϒͳײ৘ ʢෆຬΛर͑Δʣ 2024/08/23 3-shake SRE Tech Talk #10 53
54. 2024/08/23 3-shake SRE Tech Talk #10 54
55. 2024/08/23 3-shake SRE Tech Talk #10 55
56. Prompt Injection Scanner 2024/08/23 3-shake SRE Tech Talk #10 56
57. 2024/08/23 3-shake SRE Tech Talk #10 57
58. Monitorͷઃఆ 2024/08/23 3-shake SRE Tech Talk #10 58
59. 2024/08/23 3-shake SRE Tech Talk #10 59
60. SLI / SLO 2024/08/23 3-shake SRE Tech Talk #10 60
61. 2024/08/23 3-shake SRE Tech Talk #10 61
62. μογϡϘʔυ ʢσʔλྔతʹެࣜͷ৘ใΛݟͨ΄͏͕ϫΫϫΫ͠·͢ʣ 2024/08/23 3-shake SRE Tech Talk #10 62
63. 2024/08/23 3-shake SRE Tech Talk #10 63
64. 2024/08/23 3-shake SRE Tech Talk #10 64
65. 2024/08/23 3-shake SRE Tech Talk #10 65
66. 2024/08/23 3-shake SRE Tech Talk #10 66
67. 2024/08/23 3-shake SRE Tech Talk #10 67
68. ײ૝ • ෆ۩߹΍վળ఺ͷௐࠪͰ໰୊ͷཁૉΛݟఆΊΔॿ͚ʹͳΔͱײͨ͡ • ex. • ҙਤ͠ͳ͍Ԡ౴͕͋ͬͨࡍʹର৅ͷεύϯΛਂ۷Δ • Ϟσϧ͝ͱʹύλʔϯ͕ແ͍͔ಛఆ͢Δ • Ϣʔβ͔Βಧ͍ͨ໰୊͋ΔԠ౴Λಛఆ͠ௐࠪ͢Δ • τʔΫϯ΍ϨΠςϯγʹ͍ͭͯ؂ࢹͰ͖ͯ҆৺ • Ԡ౴ΛධՁͯ͠τϨʔε΍μογϡϘʔυͰ֬ೝ͢Δ͜ͱ΋Մೳ • ࣮ΞΫηεͰ͸ͳ͍͕ݕ஌͍ͨ͠৘ใ͕ݕ஌Ͱ͖ͨ • ධՁ෦෼ͷϩδοΫ͕ϒϥοΫϘοΫε͔ͭϕʔλͳͷͰࣗ෼Ͱ৘ใΛ൑அ͢Δඞཁ͸΋ͪΖΜ͋Γ • LLM ObservabilityʹݶΒͣػց຋༁΍ChatGPTࣗମΛར༻͍ͯ͠Δͱ͖΋ಉ͡ؾ࣋ͪ • Ԡ౴ͷ੒ޭ΍ϨΠςϯγͰSLOఆٛͯ͠αʔϏεͷঢ়ଶΛோΊΔΠςϨʔγϣϯͷ։࢝Ͱ͖ͦ͏ 2024/08/23 3-shake SRE Tech Talk #10 68
69. ·ͱΊ ! 2024/08/23 3-shake SRE Tech Talk #10
70. ·ͱΊ • 5,6݄ࠒ͔ΒLLM O11yʹؔ͢Δ৘ใ͕ൃੜ࢝͠Ί͍ͯΔ • ίετɺϨΠςϯγɺϨʔτϦϛοτͷΑ͏ͳϝτϦΫε͸Ϋϥ΢υ΍SaaSಉ༷େࣄ • LLMͷԠ౴ΛͲ͏ධՁ͢Δ͔ʁ͸Ҿ͖ଓ͖ൃల͍ͯ͘͠ೝࣝ • ޻෉͠ͳ͕Βࣗ෼ͨͪͷαʔϏεͰൃੜͨ͠ࣄ৅Λཧղ͠આ໌Ͱ͖Δঢ়ଶΛ໨ࢦ͢ • ܭ૷Ҏ֎Ͱମݧ΍ਫ਼౓ΛΧόʔ͢Δ޻෉΋େ੾ • Datadog LLM Observabilityʹ৮ΕͯΈͨ • ΈΜͳͷLLM O11yʹର͢Δظ଴΍͜͏ͨ͠Βྑͦ͞͏͕஌Γ͍ͨʢ࠙਌ձͰ࿩͠·͠ΐ͏ʣ 2024/08/23 3-shake SRE Tech Talk #10 70