Suggested Topicの質問に可能な限り答えてみた

Transcript

1. Japan Datadog User Group Meetup#1 ABEJA, Inc. Shogo Muranushi Suggested Topicͷ࣭໰ʹՄೳͳݶΓ౴͑ͯΈͨ
2. @rwle1212 2
3. @rwle1212 3 ໊લ ɿ ଜओ ૖ޛʢΉΒ͵͠ ͠ΐ͏͝ʣ ձࣾ ɿ ABEJA, Inc. ۀ຿ ɿΠϯϑϥΤϯδχΞ -> ϓϩμΫτΦʔφʔ -> SRE
4. @rwle1212 4 Datadogྐྵ 2೥΄ͲɻInfra ͱ Logs ͷར༻͕ϝΠϯɻ Datadog͸ࡉ͔͍ػೳ͕ଟ͘ɺͿͬͪΌ͚Α͘෼͔ͬͯͳ͍ɻ ࠓ೔࿩͢͜ͱ΋͋͘·Ͱฐࣾͷ࢖͍ํͳͷͰਖ਼ղͰ͸ͳ͍ɻ ৭ʑ஌Γ͍ͨͷͰΈΜͳͰ৘ใ࣋ͪدΓͨ͘ɺ
 ·ͣ͸ొஃͯ͠Έͨɻ
5. @rwle1212 5
6. @rwle1212 6
7. @rwle1212 7 Datadogར༻ن໛ײ docker.containers.running Datadog Logs by weekly
8. @rwle1212 8 ຊ୊ʹೖΓ·͢
9. @rwle1212 9 ࢖͍ͬͯΔػೳ΍ͦͷλΠϛϯά
 (ʮৼΓฦΓͰ͜ΜͳμογϡϘʔυΛ࢖͍ͬͯΔʯͳͲ) • Α͘͞ΘΔػೳ͸LogsͱNotebooks • ཧ༝΍λΠϛϯά͸ޙड़
10. @rwle1212 10 μογϡϘʔυͲ͏΍ͬͯઃܭͯ͠·͔͢ʁ ΅͘ͷ͔Μ͕͍͖͑ͨ͞ΐ͏ͷμογϡϘʔυ ͲΜͳDashboardͰӡ༻͍ͯ͠Δ͔ɻ εϚʔτͳμογϡϘʔυͷ࡞Γํ μογϡϘʔυ͸׶͑ͯ࢖ͬͯ·ͤΜɻ ཧ༝͸ʮ΅͘ͷ͔Μ͕͍͖͑ͨ͞ΐ͏ͷμογϡϘʔυʯʹͳͬͯ͠·͏͔ΒͰ͢ɻ
11. @rwle1212 11 ࣦഊྫ1: ݁ہCloudWatchΛݟ͍ͯͨ ALB -> Container -> EC2 -> RDSͱ͍͏ϦΫΤετͷྲྀΕͰμογϡϘʔυΛ࡞ͬ ͯΈͨɻ μογϡϘʔυͷେ͖͞͸ܾ·͍ͬͯΔɺ׌ͭάϥϑ͕ଟ͗ͯ͢΋ෳࡶʹͳΔͷ ͰɺεϚʔτʹগͳΊʹάϥϑΛ഑ஔͨ͠ɻ ͔͠͠ো֐࣌ʹ৘ใ͕଍Γͳ͍ͷͰɺ݁ہCloudWatchΛோΊ͍ͯͨ͜ͱʹؾ෇͍ ͨɻ ͔ͦ͜ΒඞཁͳάϥϑΛ଍͍ͯ͘͠ͱͲΜͲΜෳࡶʹͳΓɺଞͷϝϯόʔ͸ʮԿΛ Ͳ͏ݟͨΒྑ͍ͷ͔Θ͔Βͳ͍ʯঢ়ଶʹͳͬͨɻ ݁ہCloudWatchΛோΊ͍ͯͨ͜ͱʹؾ෇͍ͨɻ
12. @rwle1212 12 ࣦഊྫ2: 
 μογϡϘʔυͷछྨ͕૿͑ΔͱΑ͘Θ͔Βͳ͘ͳΔ
13. @rwle1212 13 ղܾྫ
 NotebookΛ࢖ͬͯΈͨ • ো֐࣌ɺௐࠪ࣌ʹඞཁͳಛఆͷάϥϑ ͕഑ஔ͞Ε͍ͯΔෛՙ֬ೝ༻ͷ NotebookΛ࡞੒ • ܦա؍࡯ΛݟΔͨΊͷάϥϑ͕഑ஔ͞ Ε͍ͯΔNotebookΛ࡞੒ ͳͲ
14. @rwle1212 14 ղܾྫ
 NotebookΛ࢖ͬͯΈͨ • Jupyter Notebook Likeͳ࢖͍ํ͕Ͱ͖ ͯɺίϝϯτͱάϥϑΛॻ͚Δ • ಛఆͷো֐ʹରͯ͠ௐ΂ͨ͜ͱɺਪଌ ͳͲͷίϝϯτΛॻ͖ͳ͕Βه࿥͕࢒ ͤΔ • ϩάͷάϥϑԽ΋༰қ • Issueʹը૾ΛషΓ෇͚ΔΑΓεϜʔ ζʹ·ͱΊΒΕΔͷͰڞ༗ָ͕͘͢͝
15. @rwle1212 15 ղܾྫ
 NotebookΛ࢖ͬͯΈͨ • ͭ·Γ൚༻తͳμογϡϘʔυ͸࡞੒ͤͣʹɺCloudWatchͳͲͷશମΛ၆ᛌͰ ͖ΔϝτϦΫεΛݟͭͭɺϩάΛάϥϑԽͨ͠Γͯ͠ɺಛఆͷো֐ʹ߹Θͤͯ ඞཁͳάϥϑΛ഑ஔͨ͠NotebookͰௐࠪ΍ܦա؍࡯Λߦ͍ͬͯΔ
16. @rwle1212 16 ͦ΋ͦ΋Έͳ͞ΜͲͷػೳԡ͠? ͱ͍͏͜ͱͰNotebooksΛਪͯ͠·͢ɻ ͋ͱLogs΋ྑ͍ࢠͰ͢ɻޙड़Ͱ·ͱ·͍ͬͯ·͢ɻ
17. @rwle1212 17 ௿ϦΫΤετ࣌ͷrateΞϥʔτ཈੍ํ๏ʹ͍ͭͯ ௿ϦΫΤετͷ؂ࢹ͸ͯ͠·ͤΜ
18. @rwle1212 18 terraformҎ֎Ͱͷ؅ཧํ๏ ओʹຊ൪༻͸TerraformɻͱΓ͋͑ͣͷ࣌͸खͰ࡞Δɻ खͰ࡞ͬͨͷͱTerraformͰ࡞ͬͨ΋ͷ͕ࠞࡏ͢Δͱɺຊ൪༻ͳͷ͔ɺΞϥʔτΛ ์ஔͯ͠ྑ͍ͷ͔Α͘෼͔Βͳ͘ͳΔͷͰʮCreateByTerraformʯTagΛ෇༩ͯ͠൑ ผ͍ͯ͠Δ
19. @rwle1212 19 ՝ۚ؅ཧʹؔ͢Δେٕখٕ ಛʹେٕ͸ແ͠ɻখωλ͕͋Δͱͨ͠Β • Multi Org؀ڥ͸Billing͕ू໿͞Εͯ͠·͏ɻʮLast Month’s UsageʯͰར༻ྔʢྉ ۚͰ͸ͳ͍ʣ͕Orgຖʹग़ྗ͞ΕΔͷͰ҈෼͍ͯ͠ΔɻӦۀ͞Μʹݴ͑͹Billing΋ ෼͚ͯ͘ΕΔ͔΋ʁखؒͳͷͰͦ͜·Ͱ΍ͬͯͳ͍ • AWSΠϯςάϨʔγϣϯͰͨ͘͞ΜͷαʔϏεͷϝτϦΫεΛऔΕΔΑ͏ʹ͢Δ ͱɺCloudWatch ͷ GetMetricData API ͕ΊͪΌͪ͘Όݺͼग़͞ΕΔͷͰ՝͕ۚ େ͖͘૿͑Δɻ·ͣ͸ඞཁͳαʔϏε͚ͩߜͬͯऔΓࠐΉͷ͕٢ɻ
 SQSͰࢮ๢ͨ͠ܦݧ༗Γ
20. @rwle1212 20 ՝ۚ؅ཧʹؔ͢Δେٕখٕ • Logs͸ʮͲΕ͕Կ݅ʯͱ͔ݟΕΔͷͰɺͨ·ʹෆཁͳϩά͕ແ͍͔֬ೝͯ͠ ʮIndexsʯͰআ֎ϑΟϧλΛ࡞੒ͯ͠ϩάͷ՝ۚΛ཈͑ͨΓ͢Δ
 ʢIndexed Logsͷ՝ۚ͸ࢭ·Δ͚ͲɺIngested Logsͷ՝ۚ͸ࢭ·Βͳ͍ͷͰ஫ҙɻ͍͔҆Βؾʹ͢Δ ΄ͲͰ͸ͳ͍͚Ͳʣ
21. @rwle1212 21 DBϞχλϦϯάͷTips͋Ε͜Ε CloudWatchΛIntegrationͯ͠ΔҐͰͦΕҎ্ͷ͜ͱ͸͓ͯ͠Γ·ͤΜ
22. @rwle1212 22 GoogleAnalyticsAPIར༻ͨ͠ݱࡏϢʔβʔ਺ͱ͔࠾औͯ͠ ͍Δࣄྫ͕͋Ε͹ GoogleAnalyticsAPI͸࢖༻͓ͯ͠Γ·ͤΜ
23. @rwle1212 ࡢ೥ͷ12݄ʹҎԼͷهࣄΛॻ͖·ͨ͠ 23 ଞͷ؂ࢹSaaS౳ʢࣗ෼͕஌ΔൣғͰ͸Zabbixɾ PrometheusɾNew RelicɾMackerelʣͱͷൺֱ
24. @rwle1212 24 ʲཁ໿ʳͳͥPrometheusΛࣙΊͯDatadogΛ࠾༻ͨ͠ͷ͔ • Datadog͔ΒPrometheus΁৐Γ׵͑ • ݩʑDatadogΛ࢖͍ͬͯͨɻ$18/host͕ߴ͔ͬͨ͠ɺKubernetesͷ؂ࢹͷσϑΝΫ τʹͳΓͭͭ͋ΔPrometheusΛࢼ͢໨త΋ؚΊͯ࢖͍࢝Ίͨɻ • ൒೥ܦաɻPrometheusࣗମͷো֐͕νϥϗϥ • OSSͰ৘ใগͳ͍৔߹͸ௐࠪʹ͕͔͔࣌ؒΔɻଟ͍࣌͸िʹ1,2ճ΄Ͳ໰୊͕ൃੜ ͠1,2ਓ͕਺࣌ؒ/ि΄ͲऔΒΕ͍ͯͨ • ελʔτΞοϓ͸ਓ͕͍ͳ͍ɻຊۀͷ։ൃʹूத͍ͨ͠ • πʔϧͷ͓कΓʹ͕࣌ؒऔΒΕ͍ͯͨɻPrometheusͷӡ༻Λ֎෦ʹҕୗ͍ͨ͠ɻ ӡ༻Λ֎෦ʹҕୗ͢Δͱ͍͘ΒʹͳΔ͔෼͔Βͳ͍͚Ͳɺࠓͷن໛ΛDatadogʹม ͑Δͱ10ສ/݄͘Β͍ͩͬͨɻɾɾDatadogͰΑ͘Ͷʁͱࢥ͍࢝ΊΔ
25. @rwle1212 25 ʲཁ໿ʳͳͥPrometheusΛࣙΊͯDatadogΛ࠾༻ͨ͠ͷ͔ • ϩάͷऩूͲ͏͢Δ໰୊ • CloudWatch LogsΛ࢖͍͕ͬͯͨݕࡧੜօແɻʢ౰࣌͸ʣPrometheus͸ϩάऩू ͢Δػೳ͸ͳ͍ɻσϑΝΫτతʹ͸ ElasticSearch + Kibana ͩΖ͏ • ͔͠͠ɺͦͷ࣌఺Ͱϩάྔ͕5ԯ/݄ɻঃʑʹର৅ͷΞϓϦ΍νʔϜΛ૿΍ͦ͏ͱࢥ ͏ͱ਺ϲ݄Ͱഒʑʹ૿͍͑ͯ͘͜ͱ͕૝ఆ͞Ε͍ͯͨ • ͦΕΛElasticSearchͰड͚ͳ͕Βӡ༻͢Δ͜ͱΛ૝ఆ͢Δͱɺख͕औΒΕͦ͏ײ͕ ൒୺ͳ͔ͬͨ • Datadog΋ϩάͷαʔϏεΛఏڙ࢝͠Ί͍ͯͨͷͰ࢖ͬͯΈΔ͜ͱʹ
26. @rwle1212 26 ʲཁ໿ʳͳͥPrometheusΛࣙΊͯDatadogΛ࠾༻ͨ͠ͷ͔ • ϩάͲ͏͢Δ͔ͷ݁࿦ • Facets ͕ڧྗͰɺಋೖָ͕ɺίετίϯτϩʔϧ΋͠΍͘͢ɺԿ͔͋Ε͹શϩά Λ௥͑Δɻͦͷ্Ͱɺϩάऩू͢Δ্ͰΠϯϑϥΛؾʹ͠ͳͯ͘ྑ͍ͷ͕ؾʹೖ ΓɺঃʑʹϩάΛ Datadog Logs ʹدͤ·ͨ͠ • ͦͯ͠ɺϞχλϦϯά΋ Promethes ͔Β Datadog ΁Ҡߦ • ͳͥPrometheusΛࣙΊͯDatadogΛ࠾༻ͨ͠ͷ͔ɺͷ·ͱΊ • ϩά΋ϞχλϦϯά΋ ͦΕ͕ࣄۀͷίΞͰ͋Ε͹ͪΌΜͱߏங͠ӡ༻͢Δ͜ͱࣗ ମ͸ԀΘͳ͍ͷͰ͕͢ɺίΞͰͳ͍ͳΒ࣌ؒΛׂ͖ͨ͘ͳ͍
27. @rwle1212 ZabbixɺPrometheus ๻ͷ؍఺͔Β͢ΔͱɺػೳੑͰ͸ͳ͘ӡ༻ੑͰ൑அ͍ͯ͠ΔͷͰOSSܥ͸Α΄ Ͳಛ௃తͳػೳ΍ཁ͕݅ͳ͍ݶΓੵۃతͳ࠾༻͸ආ͚͍ͨͰ͢ 27 ଞͷ؂ࢹSaaS౳ʢࣗ෼͕஌ΔൣғͰ͸Zabbixɾ PrometheusɾNew RelicɾMackerelʣͱͷൺֱ
28. @rwle1212 New RelicɺMackerel ྆ํͱ΋௚઀ݕূ͸͍ͯ͠ͳ͍ͷͰൺֱʹ͸ͳΒͳ͍Ͱ͕͢ɺݕূ͠ͳ͔ͬͨ ཧ༝ͷҰͭͱͯ͠ɺDatadogͰ͸ʮMonitoring Modern Infrastructureʯͱ͍͏ه ࣄ͕ެ։͞Ε͍ͯ·͢ɻ
 ͜͜ʹॻ͔Ε͍ͯΔ಺༰͕ͱͯ΋ཧʹ͔ͳ͍ͬͯͨͷͰɺ͜ͷΑ͏ͳࢥ૝ʹ ΑͬͯαʔϏε͕ઃܭ͞Εࠓޙ΋αʔϏεΛ௥Ճ͍ͯ͘͠ΜͩΖ͏ͳͱࢥ͍ɺ ࠾༻ͨ͠എܠ΋͋Γ·͢ɻ 28 ଞͷ؂ࢹSaaS౳ʢࣗ෼͕஌ΔൣғͰ͸Zabbixɾ PrometheusɾNew RelicɾMackerelʣͱͷൺֱ
29. @rwle1212 29 SLOؔ࿈ͷϞχλʔ࡞੒ݕ౼Ͱࠔͬͨ͜ͱɾ޻෉ͨ͜͠ͱ ͋·Γ೉͍͠ػೳͰ͸ͳ͍ͷͰಛʹͳ͠ʁ ڧ͍ͯ͋͛ΔͳΒʮԿΛSLOͱͯ͠ܭଌ͢Δ͔ʯͷɺͦ΋ͦ΋ͷSLOͷఆ͕ٛ೉͍͠ ͘Β͍Ͱ͢ɻ
30. @rwle1212 30 Datadogͷ෍ڭ׆ಈ ʢͲͷ໾ׂͷΤϯδχΞ·Ͱ࢖͍ͬͯΔ͔ʁʣ • ޮՌ͋ͬͨ͜ͱ • Dev/Stg/ProdͷશΞϓϦͷϩάΛશ෦ू໿ͨ͠ɻ1ͭͷWebαΠτΛݟʹ ͍͘͜ͱͰશ෦ͷϩά͕ݟΕΔঢ়ଶʹͨ͠ɻࠓ·Ͱ͸؀ڥຖʹݟΔ৔ॴ͕ ෼͔ΕͯͨΓɺग़ͯͨΓग़ͯͳ͔ͬͨΓ • ͔ͦ͜ΒঃʑʹDatadogͷ࢖͍ํΛೝࣝ͠ɺMonitors΍NotebooksɺAPM ʹϝϯόʔͷڵຯ͕޿͕ͬͯߦͬͨ
31. @rwle1212 31 Datadogͷ෍ڭ׆ಈ ʢͲͷ໾ׂͷΤϯδχΞ·Ͱ࢖͍ͬͯΔ͔ʁʣ • ޮՌ͋·Γͳ͔ͬͨ͜ͱ • Datadogͷ࢖͍ํษڧձɻʮ΁ʔɻ͜Μͳ͜ͱͰ͖ΔΜͩʯͱ͍͏͚ͩͰखΛ ಈ͔͞ͳ͍ͷͰฉ͍ͯ΋๨ΕΔ • ϩάɺϝτϦΫεΛՄೳͳݶΓू໿ͯ͠ɺඞཁʹۦΒΕͯࣗ෼Ͱௐ΂ͯ޻ ෉ͯ͠΋Β͏ํ͕͕֮͑ྑͦ͞͏ͳؾ͕ͨ͠ɻݴ͏ͯ΋೉͘͠ͳ͍ͷͰ • ͋Δఔ౓ͨ͠Βʮ͓Εͷ͔Μ͕͍͖͑ͨ͞ΐ͏ͷDatadogڞ༗ձʯΛ֤ϝ ϯόʔͰখωλతʹ΍Δͷ΋ѱ͘͸ͳ͔ͬͨ
32. @rwle1212 32 Datadogͷ෍ڭ׆ಈ ʢͲͷ໾ׂͷΤϯδχΞ·Ͱ࢖͍ͬͯΔ͔ʁʣ • Ͳͷ໾ׂͷΤϯδχΞ·Ͱ࢖͍ͬͯΔ͔ • ࢖͍ͬͯΔͷ͸ΠϯϑϥɺόοΫΤϯυ • ϑϩϯτΤϯυͷE2E؂ࢹܥ͸গ͠ߴ͍ͷͰ࢖͍ͬͯͳ͍ • Real User MonitoringɺSynthetic Monitoring (Browser)
33. @rwle1212 33 ΠϯςάϨʔγϣϯͰ޻෉͍ͯ͠Δ͜ͱʢSlack / JIRAͱ૊ Έ߹ΘͤͯɺΠϯγσϯτ؅ཧɾ໰୊؅ཧʹར༻ͳͲʣ Warning͸ʮSlack௨஌ʯ Alert͸ʮSlack௨஌ʯ+ʮPagerDutyͰి࿩ʯ+ʮGitHub Issue࡞੒(Zapierܦ༝)ʯ
34. @rwle1212 34 Ξϥʔτͷ௨஌Ϩϕϧ؅ཧ ʢฐࣾ͸warn / error / fatal ͳͲʣ • WarnͱErrorͷΈ • ʮαʔϏεʹ௚઀Өڹ͕Ͱ͍ͯΔϞϊɺͰ͍ͯΔՄೳੑ͕͋ΔϞϊʯΛΞϥʔτ ͱͯ͠ઃఆ͍ͯ͠Δ • ྫʣCPU΍ϝϞϦͷ࢖༻཰Λ؂ࢹͱ͔͸͍ͯ͠ͳ͍ɻͦΕΒͷӨڹʹΑͬ ͯALBͰ5xx͕සൃ͢Ε͹Ξϥʔτ͕ඈͿΑ͏ʹ͍ͯ͠ΔɻDynamoDBͷ εϩοτϦϯάͱ͔
35. @rwle1212 35 datadog logsʹ͍ͭͯ࢖༻ײ΍ࣄྫ
36. @rwle1212 36 facets • JSONͰૹΕ͹উखʹύʔεͯ͘͠ΕΔ • ύʔε͞Εͨ஋Λfacets(index)Ͱ͖ΔͷͰɺϩ άͷQuery΍Group ByͳͲʹར༻Ͱ͖Δ
37. @rwle1212 37 facets • Կ͕ѱ͍ͷ͔Θ͔Βͳ͍͚ͲɺindexͰ͖ͳ͍ ͕࣌͋Δɻ୭͔ղܾํ๏Λڭ͑ͯཉ͍͠ • ͜ͷέʔεͷ৔߹͸਺஋Ҏ֎ͷจࣈྻ͕ೖͬ ͪΌͬͯΔͷ͔ͳɾɾʁͬ͟ͱோΊͨײͦ͡Μ ͳ͜ͱͳ͍͚ͲɺͦΕΛ؆୯ʹݟΔʹ͸Ͳ͏͠ ͨΒྑ͍ΜͩΖ͏ɾɾ
38. @rwle1212 38 Graph by Logs • ϩάΛάϥϑԽͰ͖Δ • facetsʹొ࿥ͨ͠஋ΛGroup Byʹࢦఆͨ͠Γɺ CountʹೖΕΔ͜ͱ΋Ͱ͖Δ • ࠨਤͷྫͰ͸ɺOrganization NameຖͷϦΫΤ ετ਺΍ฏۉԠ౴࣌ؒͳͲΛදࣔɻίϯςφID ຖʹݟͯภΓ͕ແ͍͔ݟͨΓ΋ͯ͠Δ • γεςϜܥ͚ͩͰ͸ͳ͘ΞϓϦͷར༻ঢ়گ෼ੳ ʹ΋࢖༻ՄೳͳͷͰɺΞϓϦଆͰ޷͖ͳ஋Λग़ ͤ͹৭ʑͳ؍఺Ͱ෼ੳͰ͖Δ • ᮢ஋Λઃ͚ͯΞϥʔτൃ๒Ͱ͖ΔͷͰ1ͭͷγ εςϜͰϝτϦΫεɾϩάͷ؂ࢹΛ؅ཧͰ͖Δ
39. @rwle1212 39 Graph by Logs • େ͖͍มߋΛͨ࣌͠͸εςʔλείʔυͷׂ߹ Λ֬ೝ • ઃఆํ๏ • Group By: @http.status_code • Display: Areas • Stacked as: % of overall count: *
40. @rwle1212 40 Pipelines • JSON͡Όͳ͍ϩάΛύʔεͨ͠ΓɺDatadog ্Ͱʮ͜ͷϑΟʔϧυ͸εςʔλεѻ͍͢Δʯ ͱ͍͏Α͏ͳϚοϐϯάΛߦ͏ • Nginx΍RedisͳͲҰൠతͳϛυϧ΢ΣΞͷϩά ͸IntegrationΛೖΕΕ͹উखʹύʔεͯ͘͠Ε ΔͷͰࣗ෼Ͱgrokॻ͘ඞཁ͕ͳ͘ศར • ԼਤͷΑ͏ʹܭࢉࣜΛೖΕͨΓɺผ໊ʹม͑ͨ Γ͢Δ͜ͱ΋Ͱ͖Δ
41. @rwle1212 41 Indexes • ϩάऩूͷআ֎ϑΟϧλ͕ॻ͚ΔɻIndexed LogsͰ՝ۚ͞Εͳ͘ͳΔ • 10%͚ͩऔಘ͢Δͱ͍͏͜ͱ΋ՄೳɻΫΤϦͰ ͔ͳΓߜΓࠐΉ͜ͱ͕Ͱ͖Δ • ϩάΛૹ৴͢ΔAgent/ClientଆͰίϯτϩʔϧ ͢Δ͜ͱ΋Մೳ͕ͩɺઃఆϑΝΠϧͷ഑෍౳͕ ໘౗ͳͷͰ͍ͬͨΜશ෦Datadogʹ౤͛ͯɺ IndexsͰϩάΛམͱ͢ͱ͍͏࢖͍ํ͕Φεε ϝɻʢ҆Ձ͚ͩͲIngested Logsͷ՝ۚ͸ൃੜ͢ ΔͷͰ஫ҙʣ
42. @rwle1212 42 Archives • Datadogͷϩά͸ܖ໿ظؒʢ7೔ͱ͔15೔ͱ ͔ʣ͔͠ݟΕͳ͍ • աڈϩά΋௕ظؒอଘ͍ͨ͠৔߹͸ArchivesͰ S3΍GCSʹஷΊΔ͜ͱ͕Ͱ͖ΔͷͰɺAthena ΍BigQueryͰݟΔ͜ͱ΋Ͱ͖Δ • Fluentdͱ͔ͰS3ʹૹΔ͜ͱ΋Ͱ͖Δ͚Ͳɺ Datadogͷ৔߹͸ͱΓ͋͑ͣऩू͢ΔઃఆϑΝ ΠϧΛૹ৴͢Δ͚ͩͰྑ͍ͷͰɺઃఆϑΝΠϧ ͷ͹Β·͖΍ߋ৽͕ͳ͍෼ָ
43. @rwle1212 43 Archives • Rehydrate From Archivesͱ͍͏ػೳΛ࢖͑͹ ArchivesʹೖͬͯΔσʔλΛDatadog Logsʹ໭ ͢͜ͱ͕Ͱ͖ΔΈ͍ͨʁ͚ͩͲ࢖ͬͨ͜ͱͳ͍ ͷͰ஫ҙ఺͕Θ͔ͬͯͳ͍ɻ୭͔ڭ͑ͯ
44. @rwle1212 44 datadog logsʹ͍ͭͯ࢖༻ײ΍ࣄྫ ΞϓϦͷϩάΛDatadog LogsʹೖΕ͍ͯΔͷͰɺ΄΅ຖ೔౷ܭతʹݟͯҟৗ͕ແ͞ ͦ͏͔βοͱݟ͍ͯΔɻ ඞཁ͋Β͹ਂງΓ͢ΔͨΊʹ Graph Ͱ৭ʑͳ؍఺Ͱ෼ੳͨ͠Γɺه࿥࢒࣌͢ɺܦ ա؍࡯͢Δ͸NotebooksΛ࡞ͬͯܧଓతʹ༷ࢠΛݟͨΓ͍ͯ͠Δɻ
45. @rwle1212 45 Πϕϯτ৘ใʹσϓϩΠΠϕϯτͳͲɺͲΜͳ΋ͷΛૹͬ ͍ͯΔ͔ɻ ࿈ܞͤ͞ΔπʔϧΛ૿΍͢ͷ͕ݏͳͷͰ࿈ܞ͍ͤͯ͞·ͤΜ
46. @rwle1212 46 APMؔ࿈ɺAnomalyϞχλʔͷ༗༻ͳ࢖͍ํ APMɺAnomaly͸࢖͍͜ͳ͓ͤͯΓ·ͤΜ
47. @rwle1212 47 LogsઃఆͷϕετϓϥΫςΟε • ͱΓ͋͑ͣ • શ෦ͷϩάΛ1Օॴʹू໿͢Δɻѹ౗తޮ཰্͕Δ • Pipeline • ύʔεͰ͖Δ͚ͲɺͰ͖ΔݶΓJSONͰૹ৴ͯࣗ͠ಈతʹύʔεͯ͠΋Β͏ • εςʔλεͱ͔͸Ϛοϐϯάͯ͠ɺϏϡʔϫʔͷ੨ɾԫɾ੺৭දࣔͰҰ໨ྎવʹ͢Δ • Indexes • ίετ࡟ݮ͢ΔͨΊʹݟͳ͍ϩά͸আ֎͢Δ • Archives • ೦ͷͨΊաڈϩά͸S3/GCSʹอଘ͢Δ • S3/GCSଆͰҰఆظؒޙʹࣗಈ࡟আ͢ΔΑ͏ʹϥΠϑαΠΫϧΛೖΕΔ
48. 48 ͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠