初心者でもそうじゃない人も CloudWatch を使ってみよう

いろいろな監視サービスを渡り歩いて、どのサービスも一長一短あるけれど、ひとまず CloudWatch で監視を始めるのもいいと思いました。

その経緯についてお話ししました。

Transcript

1. Katz Ueno (@katzueno) JAWS-UG ໊ݹ԰, 2023.7.11 How I ended up with CloudWatch (+α) after using several services 1 ॳ৺ऀͷਓͰ΋ɺ
 ͦ͏ͯͳ͍ਓ΋ CloudWatch Λ࢖ͬͯΈΑ͏+α
2. ࠓ೔ͷݴ͍͍ͨ͜ͱ My bottom lines of the day • ଞࣾͷΦϒβόϏϦςΟαʔϏε͸ศར • ͖ͪΜͱઃܭ͠ͳ͍ͱ௧͍໨ʹૺ͏͓͔͔ۚ͠Δ • CloudWatch ϝτϦΫεɺϩάɺAthena Ͱ݁ߏ͍͚Δ
3. ͍Ζ͍ΖαʔϏεࢼ͚ͨ͠ΕͲ ·ͣ CloudWatch ࢖͏΂͔
4. ΧϨʔ 🍛 Ϛϥιϯ🏃݄ؒ400km, ंதധ, ϥΠϒ഑৴, ಈը੍࡞ ΞϚνϡΞແઢ4ڃ (1993) / TOEIC 990 (2013) / AWS SAA (2017~) CloudFront, S3, IVS, CloudWatch 4 Katz Ueno (Χπ - ্໺উ೭) Concrete CMS ࢖͍ CMS ͱΠϯϑϥ΍ͬͯ·͢ SNS: Twitter @katzueno / Instagram @katzueno & @katzcurry / Blog: katzueno.com ίϛϡχςΟ Cncrete CMS Japan / CoderDojo ඌு / JAWS UG ໊ݹ԰ ޷͖ͳ৯΂෺: झຯ: ࢿ֨: ޷͖ͳαʔϏε:
5. AWS SAP ߹֨ ઌ݄, 3೥ӽ͠ • ࢁԼ͞ΜͷຊΛങͬͯ߹֨
 ͠·ͨ͠ʂ
6. ࠓ೔ͷΞδΣϯμ Today’s agenda • CloudWatch ·Ͱͷวྺ • ֤αʔϏεͰͷར఺ɾܽ఺
 (+ࣦഊஊ) • CloudWatch (+Athena) Ͱ͍͍ ͡ΌΜ • ͓·͚: SRE ࢧ෦ͷ঺հ Stable Di ff usion Ͱ “Agenda” Ͱੜ੒
7. Katz ͷΦϒβϏϦςΟͷεϖοΫ • 3rd Party • Mackerel (2014~): ͘͞Β VPS Ͱར༻։࢝ • New Relic (2016~): AWSར༻ऀ͸ແྉͷ࣌୅ • Datadog (2021~): • AWS (CloudWatch ྺ): 2016~ • EC2, RDS த৺ɻίϯςφ͸ະͩɻ ͭ·Γɺ͠ΐ΅͍Ͱ͢ 😅 😅 😅 ඞཁʹԠͯ͡ ࠷௿ݶ͔͠ ׆༻ͯ͠ͳ͍ 😅😅😅😅😅
8. ग़ձฤ ͸ͯͳ: mackerel
9. mackarel ͸ͯͳࣾ • 2014೥αʔϏεΠϯ • ࠃ࢈؂ࢹπʔϧ • ݟ΍͍͢؅ཧը໘
10. mackarel ͸ͯͳࣾ Katz ͸ • CMS ͷ೔ຊޠαΠτ
 (͘͞ΒVPS)΍ Nextmode ͞ΜҊ݅ • AWS σϏϡʔલ͔Βར༻։࢝
11. mackarel ͸ͯͳࣾ Katz ͸ • ղܾ͔ͨͬͨ͠՝୊
 ͦ΋ͦ΋؂ࢹ͕Ͱ͖ͳ͔ͬͨ
12. mackarel ͸ͯͳࣾ ࢖ͬͯݟ͑ͨ՝୊ • ΞϓϦͷτϨʔεݟ͑ͳ͍ • ΞϓϦ (PHP) ΍ MySQL ͷΫΤϦ ύϑΥʔϚϯεݟ͑ͳ͍ • ΤϥʔτϨʔεͰ͖ͳ͍
13. ࣍ͷεςοϓฤ New Relic
14. New Relic • 2008೥૑ۀ • AWS αʔόʔ಺ͩͬͨΒແྉ ͩͬͨ • APM (Application Performance Management) αʔϏε͕͋Δʂ
15. New Relic Katz ͸ • ͓٬͞Μ CMS Ҋ݅ • ղܾ͔ͨͬͨ͠՝୊ 
 PHP Τϥʔɾॲཧ଎౓ͷ؂ࢹ • 2015~2021 ͰϝΠϯͰ࢖͏
16. New Relic ྑ͍ͱ͜Ζ • APM ศརʂ • Τϥʔ΋Θ͔Δʂ • Ͳ͜Ͱॲཧʹ͔͔࣌ؒͬͯΔ͔Θ͔ Δʂ ෆຬͳͱ͜Ζ • ΤʔδΣϯτ͕ॏ͍ (ޙड़) • ྉۚߴ͍
17. New Relic ৽ͨͳ՝୊ (౰࣌) • AWSϢʔβʔແྉͩͬͨྉ͕ۚ༗ྉʹͳͬͨ (APM+Πϯϑϥ) • (New Relic ͕ѱ͘ͳ͍͚ΕͲ) ͓ۚΛ෷͏෼ͷରՁ͋Δʁ • ෼ੳ͢Δ͕࣌ؒແ͍ • agent ͷύϑΥʔϚϯεϩε͕େ͖͔ͬͨ (ޙड़)
18. ࣍ͦ͜͸ʂ Datadog
19. Datadog • 2010೥૑ۀ • New Relic ʹൺ΂ͯαϙʔτϓ ϥοτϑΥʔϜ͕ଟ͍ (Docker ΍ k8s )
20. Datadog Katz ͸ • ͓٬͞Μ CMS Ҋ݅ • ղܾ͔ͨͬͨ͠՝୊ 
 New Relic ͷ஗͍ΤʔδΣϯτ • 2021~ ࢖͍࢝ΊΔ
21. Datadog ྑ͍ͱ͜Ζ • ؅ཧը໘͕Έ΍͍͢ (૬ੑ?) • ΤʔδΣϯτ͕͍ܰ (?) • Τϥʔݕࡧ͕؆୯ ෆຬͳͱ͜Ζ • ͦΕͰ΋ΤʔδΣϯτ͕ॏ͍ (ޙड़) • Data Injection Ϧϛοτग़ݱ😢
22. Datadog ྑ͍ͱ͜Ζ • ؅ཧը໘͕Έ΍͍͢ (૬ੑ?) • ΤʔδΣϯτ͕͍ܰ (?) • Τϥʔݕࡧ͕؆୯ ෆຬͳͱ͜Ζ • ͦΕͰ΋ΤʔδΣϯτ͕ॏ͍ (ޙड़) • Data Injection Ϧϛοτग़ݱ😢
23. Datadog ྑ͍ͱ͜Ζ • ؅ཧը໘͕Έ΍͍͢ (૬ੑ?) • ΤʔδΣϯτ͕͍ܰ (?) • Τϥʔݕࡧ͕؆୯ ෆຬͳͱ͜Ζ • ͦΕͰ΋ΤʔδΣϯτ͕ॏ͍ (ޙड़) • Data Injection Ϧϛοτग़ݱ😢
24. Datadog ྑ͍ͱ͜Ζ • ؅ཧը໘͕Έ΍͍͢ (૬ੑ?) • ΤʔδΣϯτ͕͍ܰ (?) • Τϥʔݕࡧ͕؆୯ ෆຬͳͱ͜Ζ • ͦΕͰ΋ΤʔδΣϯτ͕ॏ͍ (ޙड़) • Data Injection Ϧϛοτग़ݱ😢 27TB!!! ͝ΊΜͳ͞ʔʔʔ͍
25. Datadog ৽ͨͳ՝୊ (౰࣌) • Agent ͷύϑΥʔϚϯε௿Լ͸ಉͩͬͨ͡ • σʔλΛૹΓ͗ͯͯ͢ɺ؂ࢹϩάग़ྗΧοτ͠ͳͪ͘Ό͍͚ͳ͔ͬͨ😅 • (MySQL ͷϩά͚ͩ O ff ʹ͢ΔઃఆͳͲ͸؆୯ʹग़དྷ͚ͨͲɾɾɾ
26. ࠷ޙͷखஈ !? Zabbix !? লུɾɾɾ
27. ΤʔδΣϯτͷ ύϑΥʔϚϯεͷ ࣦഊʹ͍ͭͯ
28. New Relic & Datadog ΤʔδΣϯτͷࣦഊ ྆αʔϏεͱ΋ѱ͘ͳ͍ (?) • ΤʔδΣϯτͷϦιʔεফඅΛܭࢉͯ͠ͳ͔ͬͨ😅 • ෛՙςετ͸ΤʔδΣϯτΠϯετʔϧલ • ຊ൪ӡ༻։࢝ͨ͠ޙʹΠϯετʔϧ • CPU, ϝϞϦΛ࠷େݶʹ׆͔ͨ͢ΊͷνϡʔχϯάΛ͍ͯͨ͠ • ຊ൪Ͱӡ༻։࢝ͨ͠ࡍʹɺෛՙΦʔόʔͰΤϥʔଟൃ😭😭😭😭😭 • ͡Ό͋ࢼݧʁ → ։ൃɾݕূ؀ڥͷͨΊʹ՝ۚΛ૿΍ͨ͘͠ͳ͍
29. New Relic & Datadog ΤʔδΣϯτͷࣦഊ ྆αʔϏεͱ΋ѱ͘ͳ͍ (?) • ΤʔδΣϯτͷෛՙΛߟ͑ͯαʔόʔνϡʔχϯά͢΂͖ͩͬͨ • αʔϏε΁ͷϩάͷ༰ྔͱͦͷίετΛߟ͑ͳ͚Ε͹͍͚ͳ͔ͬͨ
30. αʔυόʔςΟʔ؂ࢹͷ·ͱΊ
31. αʔυύʔςΟʔ؂ࢹπʔϧͷ·ͱΊ ௒ݸਓతͳέʔε • ͓͕͔͔ۚΔ • ෷͍ͬͯΔ͓ۚ෼ɺӡ༻Ͱ͖͍ͯΔ͔ෆ҆
 ʢΤϥʔൃੜ࣌͸͘͢͝ศརʂʣ • ։ൃɾݕূ؀ڥͰ࢖͑ͳ͍ (අ༻) • ΤʔδΣϯτෛՙˍσʔλྔ໰୊
32. ͦ͜Ͱؾ͍ͮͨ ɾɾɾ
33. αʔυύʔςΟʔ؂ࢹπʔϧͷ·ͱΊ ௒ݸਓతͳέʔε • ͓͕͔͔ۚΔ • ෷͍ͬͯΔ͓ۚ෼ɺӡ༻Ͱ͖͍ͯΔ͔ෆ҆
 ʢΤϥʔൃੜ࣌͸͘͢͝ศརʂʣ • ։ൃɾݕূ؀ڥͰ࢖͑ͳ͍ (අ༻) • ΤʔδΣϯτෛՙˍσʔλྔ໰୊
34. αʔυύʔςΟʔ؂ࢹπʔϧͷ·ͱΊ ௒ݸਓతͳέʔε • ͓͕͔͔ۚΔ • ෷͍ͬͯΔ͓ۚ෼ɺӡ༻Ͱ͖͍ͯΔ͔ෆ҆
 ʢΤϥʔൃੜ࣌͸͘͢͝ศརʂʣ • ։ൃɾݕূ؀ڥͰ࢖͑ͳ͍ (අ༻) • ΤʔδΣϯτෛՙˍσʔλྔ໰୊ CloudWatch ͸ैྔ՝ۚ Amazon Linux ܥ ΤʔδΣϯτΠϯετʔϧ͠΍͍͢ ςετ͠΍͍͢ }
35. CloudWatch ͋Δ͡ΌΜ + Athena
36. ͱΓ͓֮͋͑ͣ͑ͯ͜͏ • CloudWatch Metrics & Dashboard • CloudWatch Logs • CloudWatch Alarm • Athena (ELB & CloudFront ͱ͔)
37. CloudWatch Ͱे෼͔΋ʁ ࠷ॳଆඋΘͬͯΔ͠ • গͳ͍අ༻Ͱ࢝ΊΒΕΔɿैྔ՝ۚ • ͦ΋ͦ΋ඪ४ͰϝτϦΫε͍ͭͯΔ • CloudWatch ϩά -> ΦʔτεέʔϦϯάΠϯελϯ εͳͲͩͱݕࡧָ͕ • (Amazon Linux) ͸ΤʔδΣϯτΠϯετʔϧؾܰ
38. CloudWatch ͷ͍͍ͱ͜Ζ
39. CloudWatch ͷར఺: Metrics • ඪ४ϝτϦΫε͕ͱΓ͍͋͑ͣͭͯ͘Δ • ϝϞϦͳͲ͸ΧελϜϝτϦΫεͰରԠ
40. CloudWatch ͷར఺: Metrics • ౦ژϦʔδϣϯ
41. CloudWatch ͷར఺: Dashboard • Dashboard ͰϝτϦΫεΛΧελϚΠζͰ͖Δ ࣾ಺νʔϜͰ ϝτϦΫεͷ ϓϦηοτΛ ڞ༗Ͱ͖Δ
42. CloudWatch ͷར఺: Dashboard • Dashboard ͰϝτϦΫεΛΧελϚΠζͰ͖Δ ࣾ಺νʔϜͰ ϝτϦΫεͷ ϓϦηοτΛ ڞ༗Ͱ͖Δ ࠓ CDK Ͱ؆୯ʹ࡞ΕΔΑ͏ʹܭըத
43. CloudWatch ͷར఺: Logs • ϩάςΩετ৘ใΛૹΔ • ݕࡧ͕Ͱ͖Δ • άϧʔϓԽͰ͖Δ • Φʔτεέʔϧάϧʔϓͷϩά Λ·ͱΊͯݕࡧͳͲͰ͖Δ
44. CloudWatch ͷར఺: Logs • ϩάͷอ࣋ظؒΛ͏·͍͜ͱௐ੔ͯ͠஋ஈઅ໿ (S3 ΑΓ΋ׂߴͳͷͰ)
45. CloudWatch ͷར఺: Alarms • ͦ΋ͦ΋ΦʔτεέʔϦϯάͯ͠Δ࣌ͱ͔ઃఆ͠·͢ • ͡Ό͋ Alarm ઃఆͯ͠Ξϥʔτग़͠·͠ΐ͏
46. CloudWatch ͷར఺: Alarms • ౦ژϦʔδϣϯͷ஋ஈ
47. Athena ࢖ͬͯ ELB & CloudFront ϩάղੳ
48. CloudFront & ELB ϩάͷ՝୊ • gz ϑΝΠϧͨ͘͞ΜͰɺͲ͔͜ʹಥͬࠐΜͰղੳ͕ඞཁ
49. ͦ͜Ͱʂʂ Athena ࢖ͬͯ ELB & CloudFront ϩάղੳ
50. Amazon Athena ͱ͸ • S3 ͷதͷσʔλΛSQLͷΑ͏ʹݕࡧͰ͖ΔαʔϏε
51. AWS Athena Ͱϩάղੳʂ • AWS υΩϡϝϯτͰςʔϒϧఆ͕ٛ͢Ͱʹެ։͞Εͯ·͢ʂ
52. AWS Athena Ͱϩάղੳʂ • AWS υΩϡϝϯτͰςʔϒϧఆ͕ٛ͢Ͱʹެ։͞Εͯ·͢ʂ
53. AWS Athena Ͱϩάղੳʂ • SQL ΫΤϦͷྫ:
 
 7/11 ʹ CloudFront Distribution ʹΞΫηεͨ͠ IP ΞυϨεͷଟ͍ॱΛݕࡧ͢Δ SQL SELECT client_ip, x_edge_location, count(*) AS num FROM cf_log WHERE request_date = '2023-07-11' GROUP BY 1,2 ORDER BY count(*) DESC LIMIT 100;
    
54. AWS Athena Ͱϩάղੳʂ • SQL ΫΤϦͷྫ:
 
 7/11 ʹ CloudFront Distribution ʹΞΫηεͨ͠ IP ΞυϨεͷଟ͍ॱΛݕࡧ͢Δ SQL SELECT client_ip, x_edge_location, count(*) AS num FROM cf_log WHERE request_date = '2023-07-11' GROUP BY 1,2 ORDER BY count(*) DESC LIMIT 100;
    
55. AWS Athena Ͱϩάղੳʂ • ౦ژϦʔδϣϯͷྉۚ σʔλ͕ଟ͍ͱ༹͓͕͚ۚΔʂʂ S3 ϥΠϑαΠΫϧઃఆͯ͠
56. ࠓ೔ͷݴ͍͍ͨ͜ͱ My bottom lines of the day • ଞࣾͷΦϒβόϏϦςΟαʔϏε͸ศར • ͖ͪΜͱઃܭ͠ͳ͍ͱ௧͍໨ʹૺ͏͓͔͔ۚ͠Δ • CloudWatch ϝτϦΫεɺϩάɺAthena Ͱ݁ߏ͍͚Δ • ͏·͘ӡ༻͢Ε͹અ໿Ͱ͖Δʂ • ؅ཧը໘ઃܭ΋ CDK ͱ͔Ͱ࡞ΕΔ (ܭըத)
57. ࠓ೔ͷݴ͍͍ͨ͜ͱ My bottom lines of the day • ͻͱ·ͣ΍ͬͯΈΔ͜ͱ • CloudWatch Dashboard Λ࡞ͬͯΈΔ • CloudWatch Logs ʹϩάΛૹͬͯݕࡧͰ͖ΔΑ͏ʹͯ͠ΈΔ
58. ͓·͚ ͕࣌ؒ༨ΔΒ͍͠ͷͰ
59. YouTube νϟϯωϧݟͯ·͔͢ʁ JAWS UG
60. JAWS UG SRE ࢧ෦ #6 ͕໘ന͔ͬͨ https://www.youtube.com/watch?v=9w8U-jd7DqI&t=1456s
61. JAWS UG SRE ࢧ෦ #6 2023/5/29 0:00:00 Φʔϓχϯά SREࢧ෦ ΈͷΔΜ @minorun365 0:05:05 ΞΠεϒϨΠΫLT SREࢧ෦ ;Δ΍ @saramune 0:09:14 ʮΤϯδχΞϦϯάͰӡ༻Λվળ͢ΔʯͨΊͷAmazon CloudWatch׆༻ ΞϚκϯ΢ΣϒαʔϏεδϟ ύϯ߹ಉձࣾɹ௡ڷ ޫ໌ @misstuu 0:39:50 શAWSΤϯδχΞʹ๋͙ɺCloudWatch ઃܭɾӡ༻ ދͷר ΞϥΠͱ΢ϚΧπ @msy78 @HorseVictory 1:03:40 Amazon CloudWatch SyntheticsͰ࢝ΊΔ֎ܗ؂ࢹʢԾʣ SREϗʔϧσΟϯάε 佂ా ߁ฏ @boy_hap 1:12:50 CloudWatchͰόϨΔʮ܅ɺ࢓ࣄதʹyoʓtube؍ͯͨΑͶʁʯ Ϋϥ΢υηϯτϦοΫ ֯த ݯଠ࿠ @kado__gen 1:21:43 CloudWatchෳ߹ΞϥʔϜͰELBͷ5XXΛ͍͍ײ͡ʹݕ஌͠Α͏ͱͨ͠Β͏·͍͔͘ͳ͔ͬͨ࿩ k.goto @365_step_tech 1:32:36 CloudWatchΛݸਓ৘ใอޢͷ໡఺ʹ͠ͳ͍ͨΊʹ גࣜձࣾΩϟϯαʔεΩϟϯ อࡔ কฏ @shoheihosaka •
63. ΧϨʔ 🍛 Ϛϥιϯ🏃݄ؒ400km, ंதധ, ϥΠϒ഑৴, ಈը੍࡞ ΞϚνϡΞແઢ4ڃ (1993) / TOEIC 990 (2013) / AWS SAA (2017~) CloudFront, S3, IVS, CloudWatch 63 Katz Ueno (Χπ - ্໺উ೭) Concrete CMS ࢖͍ CMS ͱΠϯϑϥ΍ͬͯ·͢ SNS: Twitter @katzueno / Instagram @katzueno & @katzcurry / Blog: katzueno.com ίϛϡχςΟ Cncrete CMS Japan / CoderDojo ඌு / JAWS UG ໊ݹ԰ ޷͖ͳ৯΂෺: झຯ: ࢿ֨: ޷͖ͳαʔϏε: