ECS/Fargateの活用事例 (Ruby on Rails編)

【アソビュー×ランサーズ】AWSでのオーケストレーションツールの活用事例(EKS vs ECS)

https://lancersrecruit.connpass.com/event/187989/

Transcript

1. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) ECS/Fargateͷ׆༻ࣄྫ (Ruby on Railsฤ) ϥϯαʔζגࣜձࣾ SRE/҆ୡ ྋ
2. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) ΞδΣϯμ  ɾࣗݾ঺հ ɾݱࡏऔΓ૊ΜͰ͍Δ͜ͱ ɾECS/FargateͷϝϦοτσϝϦοτ ɾ։ൃ؀ڥΛ࡞ΔϙΠϯτ ɾStg/ຊ൪؀ڥͷߏ੒ʹ͍ͭͯ ɾSSH/࡞ۀ༻ίϯςφʮdevopsʯʹ͍ͭͯ ɾCircleCIʹΑΔίϯςφσϓϩΠ ɾECS Scheduled TasksʹΑΔόον࣮૷ʹ͍ͭͯ ɾϩά/ίϯςφ؂ࢹʹ͍ͭͯ ɾ·ͱΊ
3. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) ࣗݾ঺հ
4. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) ࣗݾ঺հ  ɾ҆ୡ ྋ(adachin) ɾLancers SRE 2018~ ɾ@adachin0817 ɾRVIRUS0817 ɾblog.adachin.me ɾwiki.adachin.me ɾOSS : ɾCakePHPͰݸਓαʔϏε։ൃத
5. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) ݱࡏऔΓ૊ΜͰ͍Δ͜ͱ
6. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) ݱࡏऔΓ૊ΜͰ͍Δ͜ͱ  ɾڈ೥ळ ɾάϧʔϓձࣾ(γΫϩϚʔέςΟϯά) ɾ͘͞ΒͷΫϥ΢υͰӡ༻ ɾॳΊͯECS/FargateʹҠߦνϟϨϯδ ɾࠓ೥4݄ʹҠߦ׬ྃ ɹɾࠓճ͸ͪ͜Βͷ࿩Λ͠·͢!! ɾݱࡏ(5݄͔Β) ɾάϧʔϓձࣾ(Lancers Agency)ɹ ɾECS/FargateͰҠߦத!!
7. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) EC2ͱൺֱͨ͠ECS/Fargate ͷϝϦοτσϝϦοτ
8. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) EC2ͱൺֱͨ͠ECS/FargateͷϝϦοτσϝϦοτ  ϝϦοτ ɾଐਓԽ͕ൃੜ͠ͳ͍ ɾImmutableͳαʔόʔͷӡ༻ ɹɾ҆ఆӡ༻ɺӡ༻ޮ཰ͷΞοϓ ɾϗετΠϯελϯεͷ؅ཧ͕ল͚Δ ɾλεΫ୯ҐͰࣗ༝ʹϦιʔεมߋՄೳ ɾεέʔϧΞ΢τͷ͠΍͢͞ σϝϦοτ ɾσόοά͕ͮ͠Β͍ ɾdocker exec΍SSH͕Ͱ͖ͳ͍ ɾλεΫͷىಈ଎౓͕஗Ί ɹɾόον࣮ߦ΋਺෼஗͘ͳΔ ɾྉ͕͓ۚߴΊ ɹɾEC2ΑΓׂߴʢ1ׂʙ2ׂʣ
9. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) ։ൃ؀ڥΛ࡞Δ ϙΠϯτ
10. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) ։ൃ؀ڥΛ࡞ΔϙΠϯτ  ɾຊ൪؀ڥͱಉ౳ͷߏ੒Λ࠶ݱ ɾELB(H2O)ίϯςφ(ϦόʔεϓϩΩγ) ɾApp(ίϯςφ) ɹɾruby:2.5.7 ɹɾRails4.2.11 ɹɾNginx ɾMySQLίϯςφ ɹɾ5.7 ɾ֤ΞϓϦέʔγϣϯͷىಈ͸SupervisorͰ؅ཧ ɾbundle install͸AppίϯςφͰϩάΠϯ࣌ʹ࣮ߦ ɹɾsupervisorctlͰΞϓϦΛrestart͢ΔΑ͏ʹ
11. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) Stg/ຊ൪؀ڥͷ ߏ੒ʹ͍ͭͯ
12. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) Stg/ຊ൪؀ڥͷߏ੒ʹ͍ͭͯ  ɾTerraformͰશϦιʔείʔυԽ ɾStg؀ڥ͸λεΫ਺͸1ͭ ɾdevopsίϯςφͷར༻ ɾDB͸RDS AuroraʹҠߦ ɾը૾͸S3ʹอଘ ɾϩά͸CloudWatch Logs ɾόον͸ECS Scheduled Tasks ɾ಺෦؂ࠪ(ϩάͷू໿) ɹɾAmazon Kinesis ɾσϓϩΠ͸CircleCIͷΈͰ࣮૷ ɾ؂ࢹ͸Datadog
13. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) SSH/࡞ۀ༻ίϯςφ ʮdevopsʯʹ͍ͭͯ
14. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) SSH/࡞ۀ༻ίϯςφʮdevopsʯʹ͍ͭͯ  ɾAppʹ͸SSH͠ͳ͍ ɾSSH/࡞ۀ༻ίϯςφ(Appͱಉ౳) ɹɾdevopsͱ໋໊ ɾίϚϯυͰRDS΁઀ଓ ɹɾdump΍restore ɾaws cliͷར༻ ɾrails consoleͰͷσόοά ɾSSMͷSession Manager ΋͋Δ͕ ɹɾखܰʹ౿Έ୆͔ΒSSHΛ͍ͨͨ͠Ί ɾVPC PeeringͰϥϯαʔζͷωοτϫʔΫ͔ΒSSHϩάΠϯ
15. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) CircleCIʹΑΔ ίϯςφσϓϩΠʹ͍ͭͯ
16. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) CircleCIʹΑΔStg/ຊ൪σϓϩΠʹ͍ͭͯ(ॳظ)  ɾCircleCIͷOrbs ɹɾDockerfileͷbuild ɹɾECR΁push ɹɾECSϦϏδϣϯͷߋ৽ ɹɾαʔϏελεΫఆٛͷߋ৽ ɹɾDBϚΠάϨʔγϣϯ ɹɾrspec ɾgit flow ɾdevelopϒϥϯν ɹɾϚʔδ͢ΔͱStgʹϦϦʔε ɾmasterϒϥϯν ɹɾϚʔδ͢Δͱຊ൪ʹϦϦʔε
17. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) APIܦ༝ͰͷStg؀ڥͷσϓϩΠ(ݱࡏ)  ɾStg؀ڥͷσϓϩΠ ɹɾgithub flow ɹɾγΣϧεΫϦϓτ ɹɹɾ೚ҙͷϒϥϯνΛࢦఆ ɾૉૣ͘σϓϩΠ͕Մೳ ɹɾ։ൃޮ཰͕Ξοϓ ɾTerraform CI؀ڥΛ࣮૷ $ sh deploy.sh ԼهͷΑ͏ʹϒϥϯνΛࢦఆ࣮ͯ͠ߦ͍ͯͩ͘͠͞ɻ (ྫ) sh deploy.sh ϒϥϯν໊ $ sh deploy.sh fix-nginx { "number" : 74, "state" : "pending", "id" : "xxxxxx-xxxxx-xxxxx-xxx-xxxxxxxx", "created_at" : "2020-08-27T06:14:43.973Z" } ϒϥϯν໊/ fix-nginx ͷStgσϓϩΠΛ։࢝͠·ͨ͠!
18. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) όονͷ࣮૷ʹ͍ͭͯ
19. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) ECS Scheduled TasksʹΑΔόον࣮૷ʹ͍ͭͯ  ɾࠓ·Ͱ ɹɾgem whenever ɹɾschedule.rb ɾݱࡏ ɹɾECS Scheduled Tasks ɹɹɾCloudWatchͷΠϕϯτϧʔϧ ɹɹɾTerraformԽ { "containerOverrides": [ { "name": "HogeWorker", "command": ["bundle","exec","rails","runner","HogeWorker.new.execute"] } ] }
20. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) ϩάʗίϯςφ؂ࢹ ʹ͍ͭͯ
21. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) ϩά/ίϯςφ؂ࢹʹ͍ͭͯ  ɾΞΫηε/ΞϓϦϩάɺRDS؂ࠪ/Τϥʔϩά ɹɾCloudWatch Logsʹ ɹɾࠓ·Ͱ ɹɹɾLambdaͰS3ʹγϯΫɹɹ ɹɹɾPythonͰ࣮૷͍ͯͨͨ͠Ίӡ༻ίετΞοϓ ɹɾࠓճ ɹɹɾAmazon Kinesis Data FirehoseͰS3ʹू໿ ɹɹɾTerraformԽ ɾ؂ࢹ ɹɾMackerel→DatadogʹҠߦ ɹɾλεΫఆٛʹdatadog-agentΛಈ࡞ͤ͞ΔΑ͏ʹ ɹɾcpu:10ɺmemory:256 TerraformԽ
22. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) ·ͱΊ
23. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) ·ͱΊ  ɾγϯϓϧͳߏ੒Ͱ࣮ݱͰ͖ͨ ɾখ͍͞αʔϏε͔ΒECS/FargateԽத ɾDockerΠϝʔδͱλεΫఆٛͰӡ༻Մೳ ɾॳΊͯίϯςφຊ൪ӡ༻͢Δʹ͸͔ͳΓϋϚΔ ɹɾϔϧενΣοΫ͕௨Βͳ͍ͱίϯςφ͕མͪͨΓ ɹɾσϓϩΠͷ࣮૷͕େม ɹɾLinuxྗ͕ࢼ͞ΕΔ ɾࠓޙ ɹɾRuby2.7.1/Rails6΁ͷόʔδϣϯΞοϓ ɹɾίϯςφͷ੬ऑੑݕ஌ (Trivy) ɹɾσϓϩΠ͕10෼͔͔ΔͷͰվળ ɹɾChatOpsͰͷϦϦʔε ɹɾDatadogͰϩά࿈ܞ ɹɾϥϯαʔζຊՈ΋ίϯςφҠߦʹνϟϨϯδ
24. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS) SREٻΉʂʂ  ECS/Fargate/PHPͰͷ؀ڥΛ͓࿩͠·͢ʂ
25. 2020/9/18 ʲΞιϏϡʔ×ϥϯαʔζʳAWSͰͷΦʔέετϨʔγϣϯπʔϧͷ׆༻ࣄྫ(EKS vs ECS)