Share
Embed
AWSサービスで実現するバッチ実行環境のコンテナ/サーバレス化/ Container service of batch execution environment realized by AWS service
AWS DevDay Tokyo 2019での発表資料です
Shoichiro Nagai(shnagai)
October 04, 2019
Transcript
92 slides
-
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. T O K Y O 2 0 1 9 . 1 0 . 0 3 - 0 4 AWSαʔϏεͰ࣮ݱ͢Δ όον࣮ߦڥͷίϯςφ/αʔόϨεԽ ӬҪ উҰ ίωώτגࣜձࣾ © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. T O K Y O 2 0 1 9 . 1 0 . 0 3 - 0 4 AWSαʔϏεͰ࣮ݱ͢Δ όον࣮ߦڥͷίϯςφ/αʔόϨεԽ ӬҪ উҰ ίωώτגࣜձࣾ G - 5 -
ࣗݾհ ӬҪউҰ ओͳ׆ಈ +BQBO$POUBJOFS%BZTW "844VNNJU ͡Ίͯͷ1)1ϓϩϑΣογϣφϧ։ൃ ܦ4:45&.4 ϚϚϦ͕࣮ફ͢Δɹίϯ ςφҠߦͷਖ਼߈๏
ίωώτΤϯδχΞϒϩάIUUQTUFDIDPOOFIJUPDPNBSDIJWFBVUIPSOBHBJT ίωώτגࣜձࣾɹΤϯδχΞ ΠϯϑϥΣϒΦϖϨʔγϣϯػցֶश ࠷ۙ !TIOBHBJ -
˞ʮӾཡʯʮར༻ऀʯϝσΟΞͱΞϓϦͷ߹ܭʢ݄݄ͷฏۉʣ ˞ʮϚϚ͚/PΞϓϦʯ݄Πϯςʔδௐɹௐࠪରɿ৷தʙ̎ࡀ̌ϲ݄ͷࢠڙΛ࣋ͭঁੑ O Λநग़ ˞*OTUBHSBNͷϑΥϩϫʔɺ'BDFCPPLͷ͍͍Ͷɺ-*/&ͷͱͩͪͷ߹ܭ ݄࣌
ϚϚϦ ΞϓϦɾ8FC 4/4 *OTUBHSBNɾ-*/&ɾ'BDFCPPL هࣄ ϚϚಉ࢜ͰΈΛ૬ஊ͠߹͏2"ίϛϡχςΟΛத৺ʹ ϢʔβʔΛ֦େ͍ͯ͠·͢ ʮϚϚϦʯͰϢʔβʔಉ͕࢜ ͲΜͲΜܨ͕͍ͬͯ·͢ ϚϚͷੜ׆ʹཱͭهࣄΛ ෯͍δϟϯϧͰ৴͍ͯ͠·͢ ϚϚ͚/P̍ΞϓϦʹબग़ ਓͷϚϚ͕બͿʮݱࡏ͍ͬͯΔΞϓϦʯʹͯɺ ߲ ଞͷϚϚʹΦεεϝ͍ͨ͠ɺೝɺ ར༻ɺརศੑɺײ Ͱ̍ҐΛ֫ಘ͠·ͨ͠ هࣄ 6,000 هࣄҎ্ ྦྷܭϑΝϯ 91 ສਓ ˞ ݄ؒӾཡ 1.5ԯճ ˞ ݄ؒར༻ऀ 650ສਓ ˞ ˞ lΈzͱzڞײzΛ࣠ʹϚϚʹدΓఴ͍ ΞϓϦɾ8FCɾ4/4ͱଟ֯తʹαʔϏεΛల։͍ͯ͠·͢ "CPVUϚϚϦ -
0 450,000 900,000 1,350,000 1,800,000 2014/4 2014/5 2014/6 2014/7 2014/8
2014/9 2014/10 2014/11 2014/12 2015/1 2015/2 2015/3 2015/4 2015/5 2015/6 2015/7 2015/8 2015/9 2015/10 2015/11 2015/12 2016/1 2016/2 2016/3 2016/4 2016/5 2016/6 2016/7 2016/8 2016/9 2016/10 2016/11 2016/12 2017/1 2017/2 2017/3 2017/4 2017/5 2017/6 2017/7 2017/8 2017/9 2017/10 2017/11 2017/12 2018/1 2018/2 2018/3 2018/4 2018/5 2018/6 2018/7 2018/8 ݄ؒߘ 150ສ݅ िʹҎ্ىಈ͢Δ ΞΫςΟϒϢʔβʔ 50 ਓʹਓ 57$. ์ө ΞϓϦ૯%-ສ ਓʹਓ ਓʹਓ ਓʹਓ ਓʹਓ ˞ ˞ʮϚϚϦʯͷग़࢈༧ఆΛઃఆͨ͠Ϣʔβʔͱɺްੜ࿑ಇলൃදʮਓޱಈଶ౷ܭʯͷग़ੜ͔Βࢉग़ ˞िʹճҎ্ىಈ͢ΔϢʔβʔ ˞ ʹग़࢈ͨ͠ϚϚͷʮਓʹਓʯ͕ϚϚϦΛར༻த ຊ࠷େڃنΛތΔϒϥϯυͱ͍ͯ͠·͢ ˞ ຊ࠷େڃنΛތΔϒϥϯυ -
w ϗετܕόον࣮ߦڥͷ՝ΛৼΓฦΔ w όονॲཧͷίϯςφԽ ʲࣄྫʳ εέδϡʔϧܥόονॲཧΛίϯςφԽͯ͠&$44DIFEVMF5BTLʹҠߦ w όονॲཧͷίϯςφ αʔόϨεԽ ʲࣄྫʳ
ػցֶशͷֶशσʔλલॲཧϑϩʔ (MVF 'BSHBUF Λ 4UFQ'VODUJPOTͰඪ४Խ ΞδΣϯμ -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. ϗετܕόον࣮ߦڥͷ՝ΛৼΓฦΔ -
ͷόοναʔό αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0
* * *) ӡ༻Cόον: cron(30 0 * * *) EC2 1ͷEC2Ͱӡ༻ cronͰεέδϡʔϧཧ ༷ʑͳόονॲཧΛू େ͖ͳίϯϐϡʔςΟϯάϦιʔε -
ߏʹͯ͠Խ αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0
* * *) ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0 * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 -
όονຖʹ"DUJWF4UBOECZʹ͢Δ αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0
* * *) # ӡ༻Cόον: cron(30 0 * * *) EC2 # αʔϏεAόον: cron(10 0 * * *) # αʔϏεBόον: cron(20 0 * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεܥόοναʔό ӡ༻ܥόοναʔό -
ࢮΜͩΒखಈͰϑΣΠϧΦʔόʔ αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0
* * *) # ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0 * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεܥόοναʔό ӡ༻ܥόοναʔό ☓ ίϝϯτΞτ ֎ͯ͠Activeʹ -
ෳͷαʔόʹશ͘ಉ͡มߋΛߦ͏ඞཁ͕͋Δ αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0
* * *) # ӡ༻Cόον: cron(30 0 * * *) EC2 # αʔϏεAόον: cron(10 0 * * *) # αʔϏεBόον: cron(20 0 * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεܥόοναʔό ӡ༻ܥόοναʔό ΞϓϦέʔγϣϯσϓϩΠ ϓϥοτϑΥʔϜͷมߋ -
ෳͷαʔόʹશ͘ಉ͡มߋΛߦ͏ඞཁ͕͋Δ αʔϏεAόον: cron(10 0 * * *) αʔϏεBόον: cron(20 0
* * *) # ӡ༻Cόον: cron(30 0 * * *) EC2 # αʔϏεAόον: cron(10 0 * * *) # αʔϏεBόον: cron(20 0 * * *) ӡ༻Cόον: cron(30 0 * * *) EC2 αʔϏεܥόοναʔό ӡ༻ܥόοναʔό ΞϓϦέʔγϣϯσϓϩΠ ϓϥοτϑΥʔϜͷมߋ όοναʔόʹ͔͔Δӡ༻ෛՙେ͖͍ -
w ϓϥοτϑΥʔϜʹมߋΛՃ͑Δϋʔυϧ͕ߴ͍ όοναʔόͰಈ͘ଟذʹΘͨΔॲཧ͕ͳ͘ಈ͘Α͏ʹߟྀ͢Δඞཁ͕͋Δ ෳαʔόʹશ͘ಉ͡มߋΛՃ͑Δඞཁ͕͋Δ ৗʹԿ͔͠Βͷॲཧ͕ಈ͍͍ͯΔͷͰϩʔϧόοΫ͕͍͠ w ӡ༻ෛՙ͕ߴ͍ όοναʔόͷΠϯϑϥઃఆมߋ DSPOొͷΦϖϨʔγϣϯεςʔδϯάͰͷςετ࣮ߦ ো࣌ͷΓ͚
όονݸผͷ αʔόϦιʔε κϯϏϓϩηε͕ɺɺɺ ϗετܕόονॲཧͷ՝ όονॲཧΛίϯςφԽ͢Δ͜ͱͰղܾ -
w ϦιʔεΛڞ༗͢Δ؍͔Β៛ີͳεέδϡʔϧཧ͕ඞཁ ݶΒΕͨϚγϯϦιʔεΛڞ༗͢ΔͷͰॏ͍ॲཧ͕ॏͳΒͳ͍Α͏ߟྀ ៛ີͳεέδϡʔϧཧ όονཧா δϣϒεέδϡʔϥͷಋೖˠδϣϒεέδϡʔϥࣗମͷӡ༻͕ൃੜ εέδϡʔϦϯάͷܦҢҰ෦ʹଐਓԽ͢Δ͜ͱ͓͖͕ͪ w අ༻͕ߴ͘ͳΓ͕ͪ
Ұ൪ϚγϯϦιʔεΛ͏ॲཧΛϕʔεʹΠϯελϯεΛܾఆ͠ৗ࣌ىಈ ԽׂຖόοναʔόΛ࡞Δͱߋʹඅ༻͕ߴ͘ͳΔ Ϋϥυͷ߹ɺඞཁͳ࣌ʹ͚ͬͨͩͷඅ༻Λ͏ܗʹ͍ͨ͠ ϗετܕόονॲཧͷ՝ -
w ϦιʔεΛڞ༗͢Δ؍͔Β៛ີͳεέδϡʔϧཧ͕ඞཁ ݶΒΕͨϚγϯϦιʔεΛڞ༗͢ΔͷͰॏ͍ॲཧ͕ॏͳΒͳ͍Α͏ߟྀ ៛ີͳεέδϡʔϧཧ όονཧா δϣϒεέδϡʔϥͷಋೖˠδϣϒεέδϡʔϥࣗମͷӡ༻͕ൃੜ εέδϡʔϦϯάͷܦҢҰ෦ʹଐਓԽ͢Δ͜ͱ͓͖͕ͪ w අ༻͕ߴ͘ͳΓ͕ͪ
Ұ൪ϚγϯϦιʔεΛ͏ॲཧΛϕʔεʹΠϯελϯεΛܾఆ͠ৗ࣌ىಈ ԽׂຖόοναʔόΛ࡞Δͱߋʹඅ༻͕ߴ͘ͳΔ Ϋϥυͷ߹ɺඞཁͳ࣌ʹ͚ͬͨͩͷඅ༻Λ͏ܗʹ͍ͨ͠ ϗετܕόονॲཧͷ՝ όονॲཧΛαʔόϨεԽ͢Δ͜ͱͰղܾ -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. ʲࣄྫ1ʳ εέδϡʔϧܥόονॲཧΛίϯςφԽͯ͠ ECS ScheduleTaskʹҠߦ -
w όονΛ࢘ΔDSPOαʔόͷӡ༻ΛΊ͍ͨ શαʔϏεڞ௨ͷ&$ ಈ͍͍ͯΔͷɺ࣌ؒ௨Γʹಈ͘͜ͱ͕ظ͞ΕΔεέδϡʔϧܥόον w ΞϓϦέʔγϣϯίϯςφԽࡁͰͦͷࢿ࢈Λ׆͔͍ͨ͠ ϕʔείʔυXFCͱڞ௨ͳͷͰίϯςφԽࡁ "84Λج൫ͱ͍ͯͬͯ͠ΔͷͰɺ"84ͰίϯςφͷόονॲཧΛ࣮ݱ͍ͨ͠ ࣾͰ%PDLFSͰͷ։ൃ͕ೃછΜͰ͖͍ͯͨ ʲίϯςφಋೖͷਖ਼߈๏ʳ
IUUQTTQFBLFSEFDLDPNTIPJDIJSPODPOGSPOUBUJPOPGDPOUBJOFSUSBOTGFS ίωώτͷόοναʔόʹର͢Δ՝ -
w ଐਓԽ͘͢͠ϨΨγʔͳӡ༻͕͍ͬͯΔ ৽نొ࣌ɺDSPOUBCͰखಈεέδϡʔϧొ ϦϦʔεલͷςετ࣮ߦ44)ܦ༝Ͱόοναʔό͔Βಈ࡞֬ೝ DSPOUBCʹొ͢ΔͷҰ෦ͷΤϯδχΞͷΈ Ұ෦͔͠શ༰ΛΒͳ͍ঢ়ଶ w ΞϓϦέʔγϣϯϦϦʔεҎ֎ۃྗมߋΛՃ͑ͳ͍Α͏ʹ γϯάϧϙΠϯτͳͷͰࢭ·ΔͱαʔϏεʹӨڹେ
ग़དྷΔ͚ͩมߋՃ͑ͣʹӡ༻͍ͨ͠ͱ͍͏ҙ͕ࣝಇ͘ ίωώτͷόοναʔόʹର͢Δ՝ -
ᶃՄൖੑͷߴ͞Λ׆͔͢͜ͱͰϓϥοτϑΥʔϜมߋͷෑډ͕Լ͕Δ εςʔδϯάڥ ϓϩμΫγϣϯڥ ։ൃڥ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ όοναʔό ࣮ߦڥ
όοναʔό ࣮ߦڥ ΞϓϦέʔγϣϯ ࣮ߦڥ ΞϓϦέʔγϣϯ ࣮ߦڥ ઃఆมߋ ઃఆมߋ σϓϩΠ σϓϩΠ σϓϩΠ σϓϩΠ ઃఆมߋίϯςφͰ݁ -
ᶃՄൖੑͷߴ͞Λ׆͔͢͜ͱͰϓϥοτϑΥʔϜมߋͷෑډ͕Լ͕Δ εςʔδϯάڥ ϓϩμΫγϣϯڥ ։ൃڥ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ ΞϓϦέʔγϣϯ όοναʔό ࣮ߦڥ
όοναʔό ࣮ߦڥ ΞϓϦέʔγϣϯ ࣮ߦڥ ΞϓϦέʔγϣϯ ࣮ߦڥ ઃఆมߋ ઃఆมߋ σϓϩΠ σϓϩΠ σϓϩΠ σϓϩΠ ઃఆมߋίϯςφͰ݁ σϓϩΠཻ͕ΞϓϦέʔγϣϯίʔυ͔Β ΞϓϦέʔγϣϯ࣮ߦج൫ͦͷͷʹมΘΔ -
w ίϯςφPODSPO -BNCEB "84#BUDIΛൺֱݕ౼ ࣌ͷΞʔΩςΫνϟબఆ࣌ͷQSPTDPOTͪ͜Βʹ ʲ"NB[PO&$44DIFEVMF5BTLͰ࣮ݱ͢ΔεϚʔτͳ%PDLFSϕʔεͷόον࣮ߦڥʳ IUUQTUFDIDPOOFIJUPDPNFOUSZ w
શͯͷཁ݅Λຬͨͨ͠ͷͰ&$44DIFEVMF5BTLͷ࠾༻Λܾఆ ͜Ε͔ΒΔͳΒཁ݅ʹΑͬͯ"84#BUDIͰ͍͍ͷ͔ େنσʔλ ࣌εέδϡʔϧىಈʹରԠ͍ͯ͠ͳ͔ͬͨ w ಋೖॳଞࣾࣄྫ͕͋·Γͳ͍ͱ͍͏ෆ҆͋ͬͨ 4"ʹ૬ஊͯ͠ΞʔΩςΫνϟͷϨϏϡʔΛड͚Δ &$4ͷӡ༻࣮͋Γݟ͕͋ͬͨͷͰಋೖΛܾఆ ٕज़બఆ -
ΞʔΩςΫνϟ ECR ECS ScheduleTask Task CloudWatch Event (time-based) Aόον:
cron(10 0 * * ? *) Bόον: cron(20 0 * * ? *) Cόον: cron(30 0 * * ? *) Tag: Release Task Task Aόον Bόον Cόον pull pull pull EC2 EC2Λߏʹ͠SPOFճආ ECSλεΫஔઓུʹଇΓɺόϥ ϯεΑ͘λεΫىಈ͞ΕΔ -
όον࣮ߦͷྲྀΕ +45 ECR ECS ScheduleTask Task CloudWatch Event (time-based)
Aόον: cron(10 0 * * ? *) Bόον: cron(20 0 * * ? *) Cόον: cron(30 0 * * ? *) Tag: Release Aόον pull EC2 -
όον࣮ߦͷྲྀΕ +45 ECR ECS ScheduleTask Task CloudWatch Event (time-based)
Aόον: cron(10 0 * * ? *) Bόον: cron(20 0 * * ? *) Cόον: cron(30 0 * * ? *) Tag: Release Task Aόον Bόον pull EC2 Bόονىಈ࣌ɺ Aόονىಈ࣌ʹग़དྷͨEC2্ͷ DockerϨΠϠΩϟογϡ͕ޮ͘ͷ Ͱߴىಈ -
CόονɺσϓϩΠ͞Εͨ৽͠ ͍DockerΠϝʔδͰόονΛىಈ σϓϩΠ࣌ʹىಈ͍ͯͨ͠Bόο νɺӨڹͳ͘ॲཧΛଓ͚Δ όον࣮ߦͷྲྀΕ +45 ECR ECS CloudWatch Event
(time-based) Aόον: cron(10 0 * * ? *) Bόον: cron(20 0 * * ? *) Cόον: cron(30 0 * * ? *) Tag: Release Task Task Bόον Cόον pull (new image) EC2 -
w طଘͷ%PDLFSΠϝʔδΛࢿ࢈ͱͯ͑͠Δ XFCͱಉ͡ίʔυϕʔεͰόονΛಈ͔͢߹DNE۟ͷॻ͚͑ͩͰ0, &$44DIFEVMF5BTLͷϝϦοτ ECR ECS CMD: /usr/sbin/httpd -DFOREGROUND CMD:
bin/cake hoge_batch Web Batch Docker Pull -
w Ϋϥελ λεΫΛಈ͔͢ϗετ &$ ͷू߹ମ 'BSHBUFͷ࣌ɺͨͩͷശ w %PDLFSΠϝʔδ &$3
w λεΫఆٛ %PDLFSίϯςφͷىಈύϥϝʔλͷఆٛ w Ϋϥελͷ4DIFEVMF5BTL $MPVE8BUDI&WFOUTͱཪͰ࿈ܞ͍ͯ͠Δ &$44DIFEVMF5BTLͰόονΛΉͨΊͷཁૉ -
w Ϋϥελ λεΫΛಈ͔͢ϗετ &$ ͷू߹ମ 'BSHBUFͷ࣌ɺͨͩͷശ w %PDLFSΠϝʔδ &$3
w λεΫఆٛ %PDLFSίϯςφͷىಈύϥϝʔλͷఆٛ w Ϋϥελͷ4DIFEVMF5BTL $MPVE8BUDI&WFOUTͱཪͰ࿈ܞ͍ͯ͠Δ λεΫఆٛΛ5FSSBGPSNͰཧ λεΫఆٛΛTerraformͰཧ -
w όοναʔόͷӡ༻͕ଐਓԽ ৽نొ࣌ɺDSPOUBCͰखಈεέδϡʔϧొ ϦϦʔεલͷςετ࣮ߦ44)ܦ༝Ͱόοναʔό͔Βಈ࡞֬ೝ DSPOUBCʹొ͢ΔͷҰ෦ͷΤϯδχΞͷΈ Ұ෦͔͠શ༰ΛΒͳ͍ঢ়ଶ w ΞϓϦέʔγϣϯϦϦʔεҎ֎ۃྗมߋΛՃ͑ͳ͍Α͏ʹ γϯάϧϙΠϯτͳͷͰࢭ·ΔͱαʔϏεʹӨڹେ
ग़དྷΔ͚ͩมߋՃ͑ͣʹӡ༻͍ͨ͠ͱ͍͏ҙ͕ࣝಇ͘ όοναʔόͷଐਓԽղফ TerraformͰཧ͢Δ͜ͱͰղܾ -
όονͷ։ൃϑϩʔ ίϯςφϕʔεͰ όονॲཧΛϩʔΧϧ։ൃ APͷϦϙδτϦͰ όονॲཧͷPR terraformͷϦϙδτϦͰ όονॲཧఆٛͷPR terraformܦ༝Ͱ ຊ൪ʹλεΫఆٛొ εςʔδϯάͷECSͰ
Ұखಈ࣮ߦ͠ಈ࡞֬ೝ terraformܦ༝Ͱ εςʔδϯάʹλεΫఆٛొ ϓϥοτϑΥʔϜมߋίϯςφͰ݁ DockerͷՄൖੑͷߴ͞Ͱͦͷ··ผڥ -
όονͷ։ൃϑϩʔ ίϯςφϕʔεͰ όονॲཧΛϩʔΧϧ։ൃ APͷϦϙδτϦͰ όονॲཧͷPR terraformͷϦϙδτϦͰ όονॲཧఆٛͷPR terraformܦ༝Ͱ ຊ൪ʹλεΫఆٛొ εςʔδϯάͷECSͰ
Ұखಈ࣮ߦ͠ಈ࡞֬ೝ terraformܦ༝Ͱ εςʔδϯάʹλεΫఆٛొ ϓϥοτϑΥʔϜมߋίϯςφͰ݁ DockerͷՄൖੑͷߴ͞Ͱͦͷ··ผڥ ։ൃσϓϩΠ֬ೝͱ͍͏։ൃϑϩʔΛ ͯ͢։ൃऀͷͰߦ͏͜ͱ͕Ͱ͖Δ -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. ʲࣄྫ2ʳ ػցֶशͷֶशσʔλલॲཧϑϩʔ(Glue + Fargate) ΛStep FunctionsͰඪ४Խ -
w (MVFͰநग़ͨ͠࠷৽ͷσʔλΛػցֶशͰ͑Δܗʹม w 1ZUIPOΛͬͨࣗવݴޠॲཧ͕ϝΠϯ ಠࣗͷϥϯλΠϜΛ͍͍ͨͷͰίϯςφԽ 4BHF.BLFSͷϏϧτΠϯίϯςφͰ͍͍ͨϥΠϒϥϦ͕ରԠ͍ͯ͠ͳ͔ͬͨ w ։ൃϑϩʔϩʔΧϧ։ൃ 13ϕʔε NBTUFSϚʔδ࣌ʹ&$3ʹ%PDLFSΠϝʔδΛQVTI
w αʔόϨεͳͷͰόον࣮ߦ࣌ͷΈ՝͕ۚൃੜ ػցֶशͷલॲཧॏΊͷॲཧ͕ଟ͍ͷͰྉۚ໘ͰԸܙΛड͚͍͢ 'BSHBUFͰߦ͏લॲཧόον -
w ॲཧຖʹεέδϡʔϧ࣮ߦ (MVFεέδϡʔϧ࣮ߦ ". 'BSHBUFεέδϡʔϧ࣮ߦ ". ࠷ॳͷॲཧ͕͚ͯ͜ޙଓόον͕ಈ͍͍͍ͯ߹ͷΈʹ͑Δ w
ॲཧຖʹΤϥʔ௨ϦτϥΠ੍ޚΛೖΕΔ (MVFͰͷϦτϥΠ੍ޚͱΤϥʔ௨ όονॲཧͷίʔυͰϦτϥΠ੍ޚͱΤϥʔ௨ wॲཧεςοϓ͕૿͑Δ΄Ͳશମ૾ΛѲ͢Δͷ͕ࠔʹͳΔ εέδϡʔϧ࣮ߦύλʔϯ -
w ॲཧຖʹεέδϡʔϧ࣮ߦ (MVFεέδϡʔϧ࣮ߦ ". 'BSHBUFεέδϡʔϧ࣮ߦ ". ࠷ॳͷॲཧ͕͚ͯ͜ޙଓόον͕ಈ͍͍͍ͯ߹ͷΈʹ͑Δ w
ॲཧຖʹΤϥʔ௨ϦτϥΠ੍ޚΛೖΕΔ (MVFͰͷϦτϥΠ੍ޚͱΤϥʔ௨ όονॲཧͷίʔυͰϦτϥΠ੍ޚͱΤϥʔ௨ wॲཧεςοϓ͕૿͑Δ΄Ͳશମ૾ΛѲ͢Δͷ͕ࠔʹͳΔ εέδϡʔϧ࣮ߦύλʔϯ ͬͱศརʹग़དྷΔํ๏ͳ͍ͷ͔ -
w ॲཧຖʹεέδϡʔϧ࣮ߦ (MVFεέδϡʔϧ࣮ߦ ". 'BSHBUFεέδϡʔϧ࣮ߦ ". ࠷ॳͷॲཧ͕͚ͯ͜ޙଓόον͕ಈ͍͍͍ͯ߹ͷΈʹ͑Δ w
ॲཧຖʹΤϥʔ௨ϦτϥΠ੍ޚΛೖΕΔ (MVFͰͷϦτϥΠ੍ޚͱΤϥʔ௨ όονॲཧͷίʔυͰϦτϥΠ੍ޚͱΤϥʔ௨ wॲཧεςοϓ͕૿͑Δ΄Ͳશମ૾ΛѲ͢Δͷ͕ࠔʹͳΔ εέδϡʔϧ࣮ߦύλʔϯ 4UFQ'VODUJPOTΛ͏͜ͱͰ࣮ݱͰ͖ͨ -
w TUBUF ࣮ߦ͢Δॲཧ୯Ґ-BNCEB &$4λεΫ (MVFFUD 5ZQF۟Ͱॲཧ༰Λࢦఆ 5BTL ͳΜΒ͔ͷॲཧ࣮ߦ $IPJDF
ೖྗʹج͖ͮॲཧذ 8BJU Ұఆ࣌ؒεϦʔϓ w TUBUFNBDIJOF TUFUFͷू߹ମ ϫʔΫϑϩʔຊମ +40/ϕʔεͷ"NB[POεςʔτϝϯτݴޠͰఆٛ ͜ͷ୯ҐͰϏδϡΞϥΠζ͞ΕΔ 4UFQ'VODUJPOTͷߏཁૉ -
"-#ϩάΛ,JCBOBͰݟΔΈΛαʔόϨεʹ Amazon Elasticsearch Service ALB S3 Bucket Lambda Event
(time-based) 5ຖʹ όέοτʹϩάΛग़ྗ ObjectCreatedτϦΨ ͰLambda͕ىಈ ElasticserachʹBulkLoad GrokͰύʔε͠ɺΠϯ σοΫεొ ΤϯδχΞɺ Kibanaܦ༝ͰϩάΛݟΔ աڈͷΠϯσοΫεΛ ఆظతʹআ -
w ·ͣ"84ͷϚωʔδυαʔϏεͰ࣮ݱग़དྷͳ͍͔ߟ͑Δ ˠ͔ͨ͠͠ΒόονॲཧͰͳͯ͘Α͍ՄೳੑඞવతʹαʔόϨεߏ w -BNCEB 'BB4 Ͱ࣮ݱग़དྷͳ͍͔Λߟ͑Δ ˠϥϯλΠϜͷཧ͕ෆཁͳͷͰΓ͍ͨॲཧͷ͜ͱ͚ͩʹઐ೦ग़དྷΔ ˠαʔόϨε͔ͭ"84Ϧιʔεͱॊೈͳ࿈ܞ͕Մೳ w
ಠࣗͷϥϯλΠϜΛͬͯόονॲཧ͕͍ͨ͠ ˠΞϓϦέʔγϣϯͷίϯςφԽ ˠ&$44DIFEVMF5BTLPS"84#BUDIΛ͏ LTͱ͍͏બࢶग़ͯ͘Δ w "84্ͷॲཧͰϫʔΫϑϩʔΛΈ͍ͨ ˠ4UFQ'VODUJPOTΛ͏ "84Ͱόον࣮ߦΛߟ͑Δ࣌ͷํ -
w ·ͣ"84ͷϚωʔδυαʔϏεͰ࣮ݱग़དྷͳ͍͔ߟ͑Δ ˠ͔ͨ͠͠ΒόονॲཧͰͳͯ͘Α͍ՄೳੑඞવతʹαʔόϨεߏ w -BNCEB 'BB4 Ͱ࣮ݱग़དྷͳ͍͔Λߟ͑Δ ˠϥϯλΠϜͷཧ͕ෆཁͳͷͰΓ͍ͨॲཧͷ͜ͱ͚ͩʹઐ೦ग़དྷΔ ˠαʔόϨε͔ͭ"84Ϧιʔεͱॊೈͳ࿈ܞ͕Մೳ w
ಠࣗͷϥϯλΠϜΛͬͯόονॲཧ͕͍ͨ͠ ˠΞϓϦέʔγϣϯͷίϯςφԽ ˠ&$44DIFEVMF5BTLPS"84#BUDIΛ͏ LTͱ͍͏બࢶग़ͯ͘Δ w "84্ͷॲཧͰϫʔΫϑϩʔΛΈ͍ͨ ˠ4UFQ'VODUJPOTΛ͏ "84Ͱόον࣮ߦΛߟ͑Δ࣌ͷํ ӡ༻όον࣮ߦͷΛ Ͱ͖Δ͚ͩ"84ʹد͍ͤͯ͘͜ͱͰ ຊདྷՁͷ͋Δ։ൃʹྗ͢Δͷ͕େࣄ -
w ·ͣ"84ͷϚωʔδυαʔϏεͰ࣮ݱग़དྷͳ͍͔ߟ͑Δ ˠ͔ͨ͠͠ΒόονॲཧͰͳͯ͘Α͍ՄೳੑඞવతʹαʔόϨεߏ w -BNCEB 'BB4 Ͱ࣮ݱग़དྷͳ͍͔Λߟ͑Δ ˠϥϯλΠϜͷཧ͕ෆཁͳͷͰΓ͍ͨॲཧͷ͜ͱ͚ͩʹઐ೦ग़དྷΔ ˠαʔόϨε͔ͭ"84Ϧιʔεͱॊೈͳ࿈ܞ͕Մೳ w
ಠࣗͷϥϯλΠϜΛͬͯόονॲཧ͕͍ͨ͠ ˠΞϓϦέʔγϣϯͷίϯςφԽ ˠ&$44DIFEVMF5BTLPS"84#BUDIΛ͏ LTͱ͍͏બࢶग़ͯ͘Δ w "84্ͷॲཧͰϫʔΫϑϩʔΛΈ͍ͨ ˠ4UFQ'VODUJPOTΛ͏ "84Ͱόον࣮ߦΛߟ͑Δ࣌ͷํ )PXͱͯ͠ίϯςφαʔόϨεͱ͍ͬͨ ΞʔΩςΫνϟΛཧղ͠ ͦΕΒΛ࣮αʔϏεʹམͱ͠ࠐΉ࣌ʹ -
w ·ͣ"84ͷϚωʔδυαʔϏεͰ࣮ݱग़དྷͳ͍͔ߟ͑Δ ˠ͔ͨ͠͠ΒόονॲཧͰͳͯ͘Α͍ՄೳੑඞવతʹαʔόϨεߏ w -BNCEB 'BB4 Ͱ࣮ݱग़དྷͳ͍͔Λߟ͑Δ ˠϥϯλΠϜͷཧ͕ෆཁͳͷͰΓ͍ͨॲཧͷ͜ͱ͚ͩʹઐ೦ग़དྷΔ ˠαʔόϨε͔ͭ"84Ϧιʔεͱॊೈͳ࿈ܞ͕Մೳ w
ಠࣗͷϥϯλΠϜΛͬͯόονॲཧ͕͍ͨ͠ ˠΞϓϦέʔγϣϯͷίϯςφԽ ˠ&$44DIFEVMF5BTLPS"84#BUDIΛ͏ LTͱ͍͏બࢶग़ͯ͘Δ w "84্ͷॲཧͰϫʔΫϑϩʔΛΈ͍ͨ ˠ4UFQ'VODUJPOTΛ͏ "84Ͱόον࣮ߦΛߟ͑Δ࣌ͷํ "84ͩͬͨΓଞͷΫϥυαʔϏεΛ ͍͔ʹ͏·͍͚ͬͯ͘Δ͔͕ ͜Ε͔ΒߋʹٻΊΒΕͯ͘ΔεΩϧͩͱࢥ͏ -
w ·ͣ"84ͷϚωʔδυαʔϏεͰ࣮ݱग़དྷͳ͍͔ߟ͑Δ ˠ͔ͨ͠͠ΒόονॲཧͰͳͯ͘Α͍ՄೳੑඞવతʹαʔόϨεߏ w -BNCEB 'BB4 Ͱ࣮ݱग़དྷͳ͍͔Λߟ͑Δ ˠϥϯλΠϜͷཧ͕ෆཁͳͷͰΓ͍ͨॲཧͷ͜ͱ͚ͩʹઐ೦ग़དྷΔ ˠαʔόϨε͔ͭ"84Ϧιʔεͱॊೈͳ࿈ܞ͕Մೳ w
ಠࣗͷϥϯλΠϜΛͬͯόονॲཧ͕͍ͨ͠ ˠΞϓϦέʔγϣϯͷίϯςφԽ ˠ&$44DIFEVMF5BTLPS"84#BUDIΛ͏ LTͱ͍͏બࢶग़ͯ͘Δ w "84্ͷॲཧͰϫʔΫϑϩʔΛΈ͍ͨ ˠ4UFQ'VODUJPOTΛ͏ "84Ͱόον࣮ߦΛߟ͑Δ࣌ͷํ ਐԽͷૣ͍֤αʔϏεΛΩϟονΞοϓ͠ ࣮ࡍʹखΛಈ͔ͯ͠ثΛ૿͢͜ͱ͕େࣄ ΫϥυωΠςΟϒͳࢥ -
Thank you! © 2019, Amazon Web Services, Inc. or its
affiliates. All rights reserved. ӬҪ উҰ ίωώτגࣜձࣾ