Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AWSサービスで実現するバッチ実行環境のコンテナ/サーバレス化/ Container ...

AWSサービスで実現するバッチ実行環境のコンテナ/サーバレス化/ Container service of batch execution environment realized by AWS service

AWS DevDay Tokyo 2019での発表資料です

Shoichiro Nagai(shnagai)

October 04, 2019

Transcript

92 slides
  1. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. T O K Y O 2 0 1 9 . 1 0 . 0 3 - 0 4 AWSαʔϏεͰ࣮ݱ͢Δ όον࣮ߦ؀ڥͷίϯςφ/αʔόϨεԽ ӬҪ উҰ࿠ ίωώτגࣜձࣾ © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. T O K Y O 2 0 1 9 . 1 0 . 0 3 - 0 4 AWSαʔϏεͰ࣮ݱ͢Δ όον࣮ߦ؀ڥͷίϯςφ/αʔόϨεԽ ӬҪ উҰ࿠ ίωώτגࣜձࣾ G - 5
  2. ࣗݾ঺հ ӬҪউҰ࿠ ओͳ׆ಈ
 +BQBO$POUBJOFS%BZTW "844VNNJU
 ͸͡Ίͯͷ1)1ϓϩϑΣογϣφϧ։ൃ ೔ܦ4:45&.4 ϚϚϦ͕࣮ફ͢Δɹίϯ ςφҠߦͷਖ਼߈๏ 


    ίωώτΤϯδχΞϒϩάIUUQTUFDIDPOOFIJUPDPNBSDIJWFBVUIPSOBHBJT ίωώτגࣜձࣾɹΤϯδχΞ
 Πϯϑϥ΢ΣϒΦϖϨʔγϣϯػցֶश ࠷ۙ  !TIOBHBJ
  3.   ˞ʮӾཡ਺ʯʮར༻ऀ਺ʯ͸ϝσΟΞͱΞϓϦͷ߹ܭ஋ʢ೥݄݄ͷฏۉ஋ʣ ˞ʮϚϚ޲͚/PΞϓϦʯ͸೥݄Πϯςʔδௐ΂ɹௐࠪର৅ɿ೛৷தʙ̎ࡀ̌ϲ݄ͷࢠڙΛ࣋ͭঁੑ O  Λநग़ ˞*OTUBHSBNͷϑΥϩϫʔ਺ɺ'BDFCPPLͷ͍͍Ͷ਺ɺ-*/&ͷͱ΋ͩͪ਺ͷ߹ܭ஋ ೥݄࣌఺ 

    ϚϚϦ ΞϓϦɾ8FC 4/4 *OTUBHSBNɾ-*/&ɾ'BDFCPPL هࣄ ϚϚಉ࢜Ͱ೰ΈΛ૬ஊ͠߹͏2"ίϛϡχςΟΛத৺ʹ ϢʔβʔΛ֦େ͍ͯ͠·͢ ʮϚϚϦʯͰϢʔβʔಉ͕࢜ ͲΜͲΜܨ͕͍ͬͯ·͢ ϚϚͷੜ׆ʹ໾ཱͭهࣄΛ ෯޿͍δϟϯϧͰ഑৴͍ͯ͠·͢ ϚϚ޲͚/P̍ΞϓϦʹબग़  ਓͷϚϚ͕બͿʮݱࡏ࢖͍ͬͯΔΞϓϦʯʹͯɺ ߲໨ ଞͷϚϚʹΦεεϝ͍ͨ͠ɺೝ஌౓ɺ
 ར༻཰ɺརศੑɺ޷ײ౓ Ͱ̍ҐΛ֫ಘ͠·ͨ͠ هࣄ਺ 6,000 هࣄҎ্ ྦྷܭϑΝϯ਺ ໿ 91 ສਓ ˞ ݄ؒӾཡ਺ ໿ 1.5ԯճ ˞ ݄ؒར༻ऀ਺ ໿ 650ສਓ ˞ ˞ l೰ΈzͱzڞײzΛ࣠ʹϚϚʹدΓఴ͍ ΞϓϦɾ8FCɾ4/4ͱଟ֯తʹαʔϏεΛల։͍ͯ͠·͢ "CPVUϚϚϦ
  4. 0 450,000 900,000 1,350,000 1,800,000 2014/4 2014/5 2014/6 2014/7 2014/8

    2014/9 2014/10 2014/11 2014/12 2015/1 2015/2 2015/3 2015/4 2015/5 2015/6 2015/7 2015/8 2015/9 2015/10 2015/11 2015/12 2016/1 2016/2 2016/3 2016/4 2016/5 2016/6 2016/7 2016/8 2016/9 2016/10 2016/11 2016/12 2017/1 2017/2 2017/3 2017/4 2017/5 2017/6 2017/7 2017/8 2017/9 2017/10 2017/11 2017/12 2018/1 2018/2 2018/3 2018/4 2018/5 2018/6 2018/7 2018/8   ݄ؒ౤ߘ਺ ໿ 150ສ݅ िʹ೔Ҏ্ىಈ͢Δ ΞΫςΟϒϢʔβʔ ໿ 50 ਓʹਓ 57$. ์ө ΞϓϦ૯%-਺ສ ਓʹਓ ਓʹਓ ਓʹਓ ਓʹਓ ˞ ˞ʮϚϚϦʯ಺ͷग़࢈༧ఆ೔Λઃఆͨ͠Ϣʔβʔ਺ͱɺްੜ࿑ಇলൃදʮਓޱಈଶ౷ܭʯͷग़ੜ਺͔Βࢉग़
 ˞िʹճҎ্ىಈ͢ΔϢʔβʔ ˞ ೥ʹग़࢈ͨ͠ϚϚͷʮਓʹਓʯ͕ϚϚϦΛར༻த ೔ຊ࠷େڃن໛ΛތΔϒϥϯυ΁ͱ੒௕͍ͯ͠·͢ ˞ ೔ຊ࠷େڃن໛ΛތΔϒϥϯυ΁
  5. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. ϗετܕόον࣮ߦ؀ڥͷ՝୊ΛৼΓฦΔ
  6. ୆ͷόοναʔό αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0

    * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 1୆ͷEC2Ͱӡ༻ cronͰεέδϡʔϧ؅ཧ ༷ʑͳόονॲཧΛू໿ େ͖ͳίϯϐϡʔςΟϯάϦιʔε
  7. ୆ߏ੒ʹͯ͠৑௕Խ αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0

    * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0 * * *) ӡ༻Cόον: cron(30 0 * * *) EC2
  8. όονຖʹ"DUJWF4UBOECZʹ͢Δ αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0

    * * *) # ӡ༻Cόον: cron(30 0 * * *) EC2 # αʔϏεAόον: cron(10 0 * * *) # αʔϏεBόον: cron(20 0 * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεܥόοναʔό ӡ༻ܥόοναʔό
  9. ୆ࢮΜͩΒखಈͰϑΣΠϧΦʔόʔ αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0

    * * *) # ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0 * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεܥόοναʔό ӡ༻ܥόοναʔό ☓ ίϝϯτΞ΢τ ֎ͯ͠Activeʹ
  10. ෳ਺ͷαʔόʹશ͘ಉ͡มߋΛߦ͏ඞཁ͕͋Δ αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0

    * * *) # ӡ༻Cόον: cron(30 0 * * *) EC2 # αʔϏεAόον: cron(10 0 * * *) # αʔϏεBόον: cron(20 0 * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεܥόοναʔό ӡ༻ܥόοναʔό ΞϓϦέʔγϣϯσϓϩΠ ϓϥοτϑΥʔϜͷมߋ
  11. ෳ਺ͷαʔόʹશ͘ಉ͡มߋΛߦ͏ඞཁ͕͋Δ αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0

    * * *) # ӡ༻Cόον: cron(30 0 * * *) EC2 # αʔϏεAόον: cron(10 0 * * *) # αʔϏεBόον: cron(20 0 * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεܥόοναʔό ӡ༻ܥόοναʔό ΞϓϦέʔγϣϯσϓϩΠ ϓϥοτϑΥʔϜͷมߋ όοναʔόʹ͔͔Δӡ༻ෛՙ͸େ͖͍
  12. w ϦιʔεΛڞ༗͢Δ؍఺͔Β៛ີͳεέδϡʔϧ؅ཧ͕ඞཁ
 ݶΒΕͨϚγϯϦιʔεΛڞ༗͢ΔͷͰॏ͍ॲཧ͕ॏͳΒͳ͍Α͏ߟྀ
 ៛ີͳεέδϡʔϧ؅ཧ όον؅ཧ୆ா 
 δϣϒεέδϡʔϥͷಋೖˠδϣϒεέδϡʔϥࣗମͷӡ༻͕ൃੜ
 εέδϡʔϦϯάͷܦҢ౳͸Ұ෦ʹଐਓԽ͢Δ͜ͱ΋͓͖͕ͪ w අ༻͕ߴ͘ͳΓ͕ͪ


    Ұ൪ϚγϯϦιʔεΛ࢖͏ॲཧΛϕʔεʹΠϯελϯεΛܾఆ͠ৗ࣌ىಈ
 ৑௕Խ΍໾ׂຖόοναʔόΛ࡞Δͱߋʹඅ༻͕ߴ͘ͳΔ
 Ϋϥ΢υͷ৔߹͸ɺඞཁͳ࣌ʹ࢖ͬͨ෼͚ͩͷඅ༻Λ෷͏ܗʹ͍ͨ͠ ϗετܕόονॲཧͷ՝୊
  13. w ϦιʔεΛڞ༗͢Δ؍఺͔Β៛ີͳεέδϡʔϧ؅ཧ͕ඞཁ
 ݶΒΕͨϚγϯϦιʔεΛڞ༗͢ΔͷͰॏ͍ॲཧ͕ॏͳΒͳ͍Α͏ߟྀ
 ៛ີͳεέδϡʔϧ؅ཧ όον؅ཧ୆ா 
 δϣϒεέδϡʔϥͷಋೖˠδϣϒεέδϡʔϥࣗମͷӡ༻͕ൃੜ
 εέδϡʔϦϯάͷܦҢ౳͸Ұ෦ʹଐਓԽ͢Δ͜ͱ΋͓͖͕ͪ w අ༻͕ߴ͘ͳΓ͕ͪ


    Ұ൪ϚγϯϦιʔεΛ࢖͏ॲཧΛϕʔεʹΠϯελϯεΛܾఆ͠ৗ࣌ىಈ
 ৑௕Խ΍໾ׂຖόοναʔόΛ࡞Δͱߋʹඅ༻͕ߴ͘ͳΔ
 Ϋϥ΢υͷ৔߹͸ɺඞཁͳ࣌ʹ࢖ͬͨ෼͚ͩͷඅ༻Λ෷͏ܗʹ͍ͨ͠ ϗετܕόονॲཧͷ՝୊ όονॲཧΛαʔόϨεԽ͢Δ͜ͱͰղܾ
  14. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. ʲࣄྫ1ʳ εέδϡʔϧܥόονॲཧΛίϯςφԽͯ͠ ECS ScheduleTaskʹҠߦ
  15. ᶃՄൖੑͷߴ͞Λ׆͔͢͜ͱͰϓϥοτϑΥʔϜมߋͷෑډ͕Լ͕Δ εςʔδϯά؀ڥ ϓϩμΫγϣϯ؀ڥ ։ൃ؀ڥ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ όοναʔό ࣮ߦ؀ڥ

    όοναʔό ࣮ߦ؀ڥ ΞϓϦέʔγϣϯ ࣮ߦ؀ڥ ΞϓϦέʔγϣϯ ࣮ߦ؀ڥ ઃఆมߋ ઃఆมߋ σϓϩΠ σϓϩΠ σϓϩΠ σϓϩΠ ઃఆมߋ͸ίϯςφ಺Ͱ׬݁
  16. ᶃՄൖੑͷߴ͞Λ׆͔͢͜ͱͰϓϥοτϑΥʔϜมߋͷෑډ͕Լ͕Δ εςʔδϯά؀ڥ ϓϩμΫγϣϯ؀ڥ ։ൃ؀ڥ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ όοναʔό ࣮ߦ؀ڥ

    όοναʔό ࣮ߦ؀ڥ ΞϓϦέʔγϣϯ ࣮ߦ؀ڥ ΞϓϦέʔγϣϯ ࣮ߦ؀ڥ ઃఆมߋ ઃఆมߋ σϓϩΠ σϓϩΠ σϓϩΠ σϓϩΠ ઃఆมߋ͸ίϯςφ಺Ͱ׬݁ σϓϩΠཻ౓͕ΞϓϦέʔγϣϯίʔυ͔Β ΞϓϦέʔγϣϯ࣮ߦج൫ͦͷ΋ͷʹมΘΔ
  17. w ίϯςφPODSPO -BNCEB "84#BUDI౳Λൺֱݕ౼
 ౰࣌ͷΞʔΩςΫνϟબఆ࣌ͷQSPTDPOT͸ͪ͜Βʹ
 ʲ"NB[PO&$44DIFEVMF5BTLͰ࣮ݱ͢ΔεϚʔτͳ%PDLFSϕʔεͷόον࣮ߦ؀ڥʳ 
 IUUQTUFDIDPOOFIJUPDPNFOUSZ  w

    શͯͷཁ݅Λຬͨͨ͠ͷͰ&$44DIFEVMF5BTLͷ࠾༻Λܾఆ
 ͜Ε͔Β΍ΔͳΒཁ݅ʹΑͬͯ͸"84#BUDIͰ΋͍͍ͷ͔΋ େن໛σʔλ౳ 
 ౰࣌͸εέδϡʔϧىಈʹରԠ͍ͯ͠ͳ͔ͬͨ w ಋೖ౰ॳ͸ଞࣾࣄྫ͕͋·Γͳ͍ͱ͍͏ෆ҆͸͋ͬͨ
 4"ʹ૬ஊͯ͠ΞʔΩςΫνϟͷϨϏϡʔΛड͚Δ
 &$4ͷӡ༻࣮੷͸͋Γ஌ݟ͕͋ͬͨͷͰಋೖΛܾఆ ٕज़બఆ
  18. ΞʔΩςΫνϟ ECR ECS ScheduleTask Task CloudWatch Event 
 (time-based) Aόον:

    cron(10 0 * * ? *) Bόον: cron(20 0 * * ? *) Cόον: cron(30 0 * * ? *) Tag: Release Task Task Aόον Bόον Cόον pull pull pull EC2 EC2Λ৑௕ߏ੒ʹ͠SPOFճආ ECSλεΫ഑ஔઓུʹଇΓɺόϥ ϯεΑ͘λεΫ͸ىಈ͞ΕΔ
  19. όον࣮ߦͷྲྀΕ +45 ECR ECS ScheduleTask Task CloudWatch Event 
 (time-based)

    Aόον: cron(10 0 * * ? *) Bόον: cron(20 0 * * ? *) Cόον: cron(30 0 * * ? *) Tag: Release Aόον pull EC2
  20. όον࣮ߦͷྲྀΕ +45 ECR ECS ScheduleTask Task CloudWatch Event 
 (time-based)

    Aόον: cron(10 0 * * ? *) Bόον: cron(20 0 * * ? *) Cόον: cron(30 0 * * ? *) Tag: Release Task Aόον Bόον pull EC2 Bόονىಈ࣌͸ɺ Aόονىಈ࣌ʹग़དྷͨEC2্ͷ DockerϨΠϠΩϟογϡ͕ޮ͘ͷ Ͱߴ଎ىಈ
  21. Cόον͸ɺσϓϩΠ͞Εͨ৽͠ ͍DockerΠϝʔδͰόονΛىಈ σϓϩΠ࣌ʹىಈ͍ͯͨ͠Bόο ν͸ɺӨڹͳ͘ॲཧΛଓ͚Δ όον࣮ߦͷྲྀΕ +45 ECR ECS CloudWatch Event

    
 (time-based) Aόον: cron(10 0 * * ? *) Bόον: cron(20 0 * * ? *) Cόον: cron(30 0 * * ? *) Tag: Release Task Task Bόον Cόον pull (new image) EC2
  22. w Ϋϥελ
 λεΫΛಈ͔͢ϗετ &$ ͷू߹ମ
 'BSHBUFͷ࣌͸ɺͨͩͷശ w %PDLFSΠϝʔδ &$3 

    w λεΫఆٛ
 %PDLFSίϯςφͷىಈύϥϝʔλͷఆٛ w Ϋϥελ಺ͷ4DIFEVMF5BTL
 $MPVE8BUDI&WFOUTͱཪͰ࿈ܞ͍ͯ͠Δ &$44DIFEVMF5BTLͰόονΛ૊ΉͨΊͷཁૉ
  23. w Ϋϥελ
 λεΫΛಈ͔͢ϗετ &$ ͷू߹ମ
 'BSHBUFͷ࣌͸ɺͨͩͷശ w %PDLFSΠϝʔδ &$3 

    w λεΫఆٛ
 %PDLFSίϯςφͷىಈύϥϝʔλͷఆٛ w Ϋϥελ಺ͷ4DIFEVMF5BTL
 $MPVE8BUDI&WFOUTͱཪͰ࿈ܞ͍ͯ͠Δ λεΫఆٛΛ5FSSBGPSNͰ؅ཧ λεΫఆٛΛTerraformͰ؅ཧ
  24. όονͷ։ൃϑϩʔ ίϯςφϕʔεͰ όονॲཧΛϩʔΧϧ։ൃ APͷϦϙδτϦͰ όονॲཧͷPR terraformͷϦϙδτϦͰ όονॲཧఆٛͷPR terraformܦ༝Ͱ ຊ൪ʹλεΫఆٛొ࿥ εςʔδϯάͷECSͰ

    Ұ౓खಈ࣮ߦ͠ಈ࡞֬ೝ terraformܦ༝Ͱ εςʔδϯάʹλεΫఆٛొ࿥ ϓϥοτϑΥʔϜมߋ͸ίϯςφͰ׬݁ DockerͷՄൖੑͷߴ͞Ͱͦͷ··ผ؀ڥ΁
  25. όονͷ։ൃϑϩʔ ίϯςφϕʔεͰ όονॲཧΛϩʔΧϧ։ൃ APͷϦϙδτϦͰ όονॲཧͷPR terraformͷϦϙδτϦͰ όονॲཧఆٛͷPR terraformܦ༝Ͱ ຊ൪ʹλεΫఆٛొ࿥ εςʔδϯάͷECSͰ

    Ұ౓खಈ࣮ߦ͠ಈ࡞֬ೝ terraformܦ༝Ͱ εςʔδϯάʹλεΫఆٛొ࿥ ϓϥοτϑΥʔϜมߋ͸ίϯςφͰ׬݁ DockerͷՄൖੑͷߴ͞Ͱͦͷ··ผ؀ڥ΁ ։ൃσϓϩΠ֬ೝͱ͍͏։ൃϑϩʔΛ ͢΂ͯ։ൃऀͷ੹຿Ͱߦ͏͜ͱ͕Ͱ͖Δ
  26. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. ʲࣄྫ2ʳ ػցֶशͷֶशσʔλલॲཧϑϩʔ(Glue + Fargate) ΛStep FunctionsͰඪ४Խ
  27. w ॲཧຖʹεέδϡʔϧ࣮ߦ
 (MVFεέδϡʔϧ࣮ߦ ". 
 'BSHBUFεέδϡʔϧ࣮ߦ ". 
 ࠷ॳͷॲཧ͕͚ͯ͜ޙଓόον͕ಈ͍ͯ΋͍͍৔߹ͷΈʹ࢖͑Δ w

    ॲཧຖʹΤϥʔ௨஌΍ϦτϥΠ੍ޚΛೖΕΔ
 (MVFͰͷϦτϥΠ੍ޚͱΤϥʔ௨஌
 όονॲཧͷίʔυͰϦτϥΠ੍ޚͱΤϥʔ௨஌ wॲཧεςοϓ͕૿͑Δ΄Ͳશମ૾Λ೺Ѳ͢Δͷ͕ࠔ೉ʹͳΔ εέδϡʔϧ࣮ߦύλʔϯ
  28. w ॲཧຖʹεέδϡʔϧ࣮ߦ
 (MVFεέδϡʔϧ࣮ߦ ". 
 'BSHBUFεέδϡʔϧ࣮ߦ ". 
 ࠷ॳͷॲཧ͕͚ͯ͜ޙଓόον͕ಈ͍ͯ΋͍͍৔߹ͷΈʹ࢖͑Δ w

    ॲཧຖʹΤϥʔ௨஌΍ϦτϥΠ੍ޚΛೖΕΔ
 (MVFͰͷϦτϥΠ੍ޚͱΤϥʔ௨஌
 όονॲཧͷίʔυͰϦτϥΠ੍ޚͱΤϥʔ௨஌ wॲཧεςοϓ͕૿͑Δ΄Ͳશମ૾Λ೺Ѳ͢Δͷ͕ࠔ೉ʹͳΔ εέδϡʔϧ࣮ߦύλʔϯ ΋ͬͱศརʹग़དྷΔํ๏͸ͳ͍ͷ͔
  29. w ॲཧຖʹεέδϡʔϧ࣮ߦ
 (MVFεέδϡʔϧ࣮ߦ ". 
 'BSHBUFεέδϡʔϧ࣮ߦ ". 
 ࠷ॳͷॲཧ͕͚ͯ͜ޙଓόον͕ಈ͍ͯ΋͍͍৔߹ͷΈʹ࢖͑Δ w

    ॲཧຖʹΤϥʔ௨஌΍ϦτϥΠ੍ޚΛೖΕΔ
 (MVFͰͷϦτϥΠ੍ޚͱΤϥʔ௨஌
 όονॲཧͷίʔυͰϦτϥΠ੍ޚͱΤϥʔ௨஌ wॲཧεςοϓ͕૿͑Δ΄Ͳશମ૾Λ೺Ѳ͢Δͷ͕ࠔ೉ʹͳΔ εέδϡʔϧ࣮ߦύλʔϯ 4UFQ'VODUJPOTΛ࢖͏͜ͱͰ࣮ݱͰ͖ͨ
  30. w TUBUF
 ࣮ߦ͢Δॲཧ୯Ґ-BNCEB &$4λεΫ (MVFFUD
 5ZQF۟Ͱॲཧ಺༰Λࢦఆ
 5BTL ͳΜΒ͔ͷॲཧ࣮ߦ 
 $IPJDF

    ೖྗʹج͖ͮॲཧ෼ذ 
 8BJU Ұఆ࣌ؒεϦʔϓ  w TUBUFNBDIJOF
 TUFUFͷू߹ମ ϫʔΫϑϩʔຊମ 
 +40/ϕʔεͷ"NB[POεςʔτϝϯτݴޠͰఆٛ
 ͜ͷ୯ҐͰϏδϡΞϥΠζ͞ΕΔ 4UFQ'VODUJPOTͷߏ੒ཁૉ
  31. "-#ϩάΛ,JCBOBͰݟΔ࢓૊ΈΛαʔόϨεʹ Amazon Elasticsearch Service ALB S3 Bucket Lambda Event 


    (time-based) 5෼ຖʹ όέοτʹϩάΛग़ྗ ObjectCreatedτϦΨ ͰLambda͕ىಈ ElasticserachʹBulkLoad GrokͰύʔε͠ɺΠϯ σοΫεొ࿥ ΤϯδχΞ͸ɺ Kibanaܦ༝ͰϩάΛݟΔ աڈ෼ͷΠϯσοΫεΛ ఆظతʹ࡟আ
  32. w ·ͣ"84ͷϚωʔδυαʔϏεͰ࣮ݱग़དྷͳ͍͔ߟ͑Δ
 ˠ΋͔ͨ͠͠ΒόονॲཧͰͳͯ͘΋Α͍Մೳੑ΋ඞવతʹαʔόϨεߏ੒ w -BNCEB 'BB4 Ͱ࣮ݱग़དྷͳ͍͔Λߟ͑Δ
 ˠϥϯλΠϜͷ؅ཧ͕ෆཁͳͷͰ΍Γ͍ͨॲཧͷ͜ͱ͚ͩʹઐ೦ग़དྷΔ
 ˠαʔόϨε͔ͭ"84Ϧιʔεͱॊೈͳ࿈ܞ͕Մೳ w

    ಠࣗͷϥϯλΠϜΛ࢖ͬͯόονॲཧ͕͍ͨ͠
 ˠΞϓϦέʔγϣϯͷίϯςφԽ
 ˠ&$44DIFEVMF5BTLPS"84#BUDIΛ࢖͏ LTͱ͍͏બ୒ࢶ΋ग़ͯ͘Δ  w "84্ͷॲཧͰϫʔΫϑϩʔΛ૊Έ͍ͨ
 ˠ4UFQ'VODUJPOTΛ࢖͏ "84Ͱόον࣮ߦΛߟ͑Δ࣌ͷํ਑
  33. w ·ͣ"84ͷϚωʔδυαʔϏεͰ࣮ݱग़དྷͳ͍͔ߟ͑Δ
 ˠ΋͔ͨ͠͠ΒόονॲཧͰͳͯ͘΋Α͍Մೳੑ΋ඞવతʹαʔόϨεߏ੒ w -BNCEB 'BB4 Ͱ࣮ݱग़དྷͳ͍͔Λߟ͑Δ
 ˠϥϯλΠϜͷ؅ཧ͕ෆཁͳͷͰ΍Γ͍ͨॲཧͷ͜ͱ͚ͩʹઐ೦ग़དྷΔ
 ˠαʔόϨε͔ͭ"84Ϧιʔεͱॊೈͳ࿈ܞ͕Մೳ w

    ಠࣗͷϥϯλΠϜΛ࢖ͬͯόονॲཧ͕͍ͨ͠
 ˠΞϓϦέʔγϣϯͷίϯςφԽ
 ˠ&$44DIFEVMF5BTLPS"84#BUDIΛ࢖͏ LTͱ͍͏બ୒ࢶ΋ग़ͯ͘Δ  w "84্ͷॲཧͰϫʔΫϑϩʔΛ૊Έ͍ͨ
 ˠ4UFQ'VODUJPOTΛ࢖͏ "84Ͱόον࣮ߦΛߟ͑Δ࣌ͷํ਑ ӡ༻όον࣮ߦͷ੹຿Λ Ͱ͖Δ͚ͩ"84ʹد͍ͤͯ͘͜ͱͰ ຊདྷՁ஋ͷ͋Δ։ൃʹ஫ྗ͢Δͷ͕େࣄ
  34. w ·ͣ"84ͷϚωʔδυαʔϏεͰ࣮ݱग़དྷͳ͍͔ߟ͑Δ
 ˠ΋͔ͨ͠͠ΒόονॲཧͰͳͯ͘΋Α͍Մೳੑ΋ඞવతʹαʔόϨεߏ੒ w -BNCEB 'BB4 Ͱ࣮ݱग़དྷͳ͍͔Λߟ͑Δ
 ˠϥϯλΠϜͷ؅ཧ͕ෆཁͳͷͰ΍Γ͍ͨॲཧͷ͜ͱ͚ͩʹઐ೦ग़དྷΔ
 ˠαʔόϨε͔ͭ"84Ϧιʔεͱॊೈͳ࿈ܞ͕Մೳ w

    ಠࣗͷϥϯλΠϜΛ࢖ͬͯόονॲཧ͕͍ͨ͠
 ˠΞϓϦέʔγϣϯͷίϯςφԽ
 ˠ&$44DIFEVMF5BTLPS"84#BUDIΛ࢖͏ LTͱ͍͏બ୒ࢶ΋ग़ͯ͘Δ  w "84্ͷॲཧͰϫʔΫϑϩʔΛ૊Έ͍ͨ
 ˠ4UFQ'VODUJPOTΛ࢖͏ "84Ͱόον࣮ߦΛߟ͑Δ࣌ͷํ਑ )PXͱͯ͠ίϯςφ΍αʔόϨεͱ͍ͬͨ ΞʔΩςΫνϟΛཧղ͠ ͦΕΒΛ࣮αʔϏεʹམͱ͠ࠐΉ࣌ʹ
  35. w ·ͣ"84ͷϚωʔδυαʔϏεͰ࣮ݱग़དྷͳ͍͔ߟ͑Δ
 ˠ΋͔ͨ͠͠ΒόονॲཧͰͳͯ͘΋Α͍Մೳੑ΋ඞવతʹαʔόϨεߏ੒ w -BNCEB 'BB4 Ͱ࣮ݱग़དྷͳ͍͔Λߟ͑Δ
 ˠϥϯλΠϜͷ؅ཧ͕ෆཁͳͷͰ΍Γ͍ͨॲཧͷ͜ͱ͚ͩʹઐ೦ग़དྷΔ
 ˠαʔόϨε͔ͭ"84Ϧιʔεͱॊೈͳ࿈ܞ͕Մೳ w

    ಠࣗͷϥϯλΠϜΛ࢖ͬͯόονॲཧ͕͍ͨ͠
 ˠΞϓϦέʔγϣϯͷίϯςφԽ
 ˠ&$44DIFEVMF5BTLPS"84#BUDIΛ࢖͏ LTͱ͍͏બ୒ࢶ΋ग़ͯ͘Δ  w "84্ͷॲཧͰϫʔΫϑϩʔΛ૊Έ͍ͨ
 ˠ4UFQ'VODUJPOTΛ࢖͏ "84Ͱόον࣮ߦΛߟ͑Δ࣌ͷํ਑ "84ͩͬͨΓଞͷΫϥ΢υαʔϏεΛ ͍͔ʹ͏·͘࢖͍͚ͬͯΔ͔͕ ͜Ε͔ΒߋʹٻΊΒΕͯ͘ΔεΩϧͩͱࢥ͏
  36. w ·ͣ"84ͷϚωʔδυαʔϏεͰ࣮ݱग़དྷͳ͍͔ߟ͑Δ
 ˠ΋͔ͨ͠͠ΒόονॲཧͰͳͯ͘΋Α͍Մೳੑ΋ඞવతʹαʔόϨεߏ੒ w -BNCEB 'BB4 Ͱ࣮ݱग़དྷͳ͍͔Λߟ͑Δ
 ˠϥϯλΠϜͷ؅ཧ͕ෆཁͳͷͰ΍Γ͍ͨॲཧͷ͜ͱ͚ͩʹઐ೦ग़དྷΔ
 ˠαʔόϨε͔ͭ"84Ϧιʔεͱॊೈͳ࿈ܞ͕Մೳ w

    ಠࣗͷϥϯλΠϜΛ࢖ͬͯόονॲཧ͕͍ͨ͠
 ˠΞϓϦέʔγϣϯͷίϯςφԽ
 ˠ&$44DIFEVMF5BTLPS"84#BUDIΛ࢖͏ LTͱ͍͏બ୒ࢶ΋ग़ͯ͘Δ  w "84্ͷॲཧͰϫʔΫϑϩʔΛ૊Έ͍ͨ
 ˠ4UFQ'VODUJPOTΛ࢖͏ "84Ͱόον࣮ߦΛߟ͑Δ࣌ͷํ਑ ਐԽͷૣ͍֤αʔϏεΛΩϟονΞοϓ͠ ࣮ࡍʹखΛಈ͔ͯ͠෢ثΛ૿΍͢͜ͱ͕େࣄ Ϋϥ΢υωΠςΟϒͳࢥ૝΁
  37. Thank you! © 2019, Amazon Web Services, Inc. or its

    affiliates. All rights reserved. ӬҪ উҰ࿠ ίωώτגࣜձࣾ

More Decks by Shoichiro Nagai(shnagai)

Other Decks in Technology