ECSとSQSでスケーラブルなバッチを作った

Transcript

1. &$4ͱ424Ͱ
   εέʔϥϒϧͳόονΛ࡞ͬͨ ٢ాوจ ![FQIJSBOTBT
   Ϋϥεϝιουגࣜձࣾ
2. εϥΠυ͸ޙͰೖख͢Δ͜ͱ͕ग़དྷ·͢ͷͰ ൃදதͷ಺༰ΛϝϞ͢Δඞཁ͸͋Γ·ͤΜɻ ࣸਅࡱӨΛ͢Δ৔߹͸ ϑϥογϡɾγϟολʔԻ͕ग़ͳ͍Α͏ʹ͝഑ྀ͍ͩ͘͞ Attention
3. 
    #jawsug #jawsoka #soracomug
4. ࣗݾ঺հ
    ٢ాوจ ![FQIJSBOTBT
   wΫϥεϝιουגࣜձࣾ
   w$9ࣄۀຊ෦αʔόαΠυΤϯδχΞ
   wԬࢁ+BWBϢʔβձ
   ୅දΦʔϓϯη ϛφʔ
   Ԭࢁ࣮ߦҕһ
   w޷͖ͳ"84ͷαʔϏε
   w&$4 %ZOBNP%#
5. ΞδΣϯμ
    wΞʔΩςΫνϟ֓ཁ
   w4XJUDI
   3PMFʹ͍ͭͯ
   w424Ͱ΍ͬͨ͜ͱ
   w&$4Ͱ΍ͬͨ͜ͱ
   w΍ͬͯΈͨ
6. 
    ΞʔΩςΫνϟ֓ཁ
7. ΍Γ͍ͨ͜ͱ
    wσʔλͷҰׅߋ৽
   wݩσʔλ͸$47
   wσʔλྔ͸े਺ສ݅ఔ౓
   w*%ͱɺߋ৽಺༰͕ೖ͍ͬͯΔ
   wߋ৽ʹ͸֎෦ͷ"1*Λୟ͘
8. 
   
9. 
    Switch RoleͰ ΍ͬͨ͜ͱ
10. 
     w424΁ͷσʔλૹ৴ॲཧͰ4XJUDI
    3PMF͍ͨ͠
    w4XJUDI
    3PMF͢Δʹ͸.'"ඞਢ
    wBXTDMJͰ͋Ε͹్தͰτʔΫϯΛೖྗͰ͖Δ
    w4%,ͩͱࣗલͰΫϨσϯγϟϧΛऔಘͯ͠΍ Δඞཁ͕͋Δ
11. BXTDPOpH
     [default] region = ap-northeast-1 output = json [profile hoge] region = ap-northeast-1 source_profile = default role_arn = arn:aws:iam::ACCOUNT_ID:role/john-doe mfa_serial = arn:aws:iam::ACCOUNT_ID:mfa/john-doe
12. BXTDMJͰ4XJUDI
    3PMF͢Δ৔߹
     $ AWS_PROFILE=hoge aws s3 ls Enter MFA code for arn:aws:iam::ACCOUNT_ID:mfa/john-doe [MFAτʔΫϯΛೖྗ͢Δ]
13. "3/ɺ.'"τʔΫϯɺTUTΫϥΠΞϯτ
     sts_client = Aws::STS::Client.new(region: 'ap-northeast-1') role_arn = `aws configure get role_arn --profile hoge`.chomp serial_number = `aws configure get mfa_serial --profile hoge`.chomp puts "Input MFA token code..." token_code = gets.chomp
14. ΫϨσϯγϟϧੜ੒͠424ΫϥΠΞϯτΛ࡞੒
     role_credentials = Aws::AssumeRoleCredentials.new( client: sts_client, role_arn: role_arn, role_session_name: "hoge_session", serial_number: serial_number, token_code: token_code) Aws::SQS::Client.new(credentials: role_credentials)
15. 
     w؀ڥม਺"84@130'*-&͸ར༻͠ͳ͍
    wBXT
    DPOpHVSF
    HFUͰඞཁͳ"3/Λऔಘ͢Δ
    w.'"τʔΫϯ͸ผ్ɺೖྗͤ͞Δ
    wTUTΫϥΠΞϯτΛ࡞੒͠ɺ "TTVNF3PMF$SFEFOUJBMTͰɺΫϨσϯγϟϧ Λऔಘ͢Δ
16. 
     SQSͰ΍ͬͨ͜ͱ
17. 
     wෳ਺ͷλεΫ͔ΒΞΫηε͞ΕΔͷͰɺ͜ΕΛ ͍͍ײ͡ʹॲཧͰ͖Δ
    wॲཧͰ͖ͳ͔ͬͨσʔλΛɺผΩϡʔʹҠͯ͠ ϦτϥΠ͠΍͘͢͢Δ
    w424ͷ%FBE
    -FUUFS
    2VFVFͷ࢓૊ΈΛ࢖͏
18. 
    
19. 
     #PEZ 3FDFJWF
    $PVOU 7JTJCMF   536&
20. 
     #PEZ 3FDFJWF
    $PVOU 7JTJCMF   '"-4&
21. 
     #PEZ 3FDFJWF
    $PVOU 7JTJCMF   536& VisibilityTimeoutΛա͗ͯ΋Delete͞Εͳ͔ͬͨ৔߹
22. 
     #PEZ 3FDFJWF
    $PVOU 7JTJCMF Receive Count͕࠷େReceive CountΛ௒͑ͨ৔߹ #PEZ 3FDFJWF
    $PVOU 7JTJCMF   536& DLQ΁Ҡಈ
23. 
     ECSͰ΍ͬͨ͜ͱ
24. 
     w'BSHBUFͰϦιʔε؅ཧͷखؒΛݮΒ͍ͨ͠
    wฒྻͰ࣮ߦͰ͖ΔΑ͏ʹ͍ͨ͠
    wঢ়گʹԠͯ͡ɺλεΫͷ਺Λௐ੔͍ͨ͠
    w$MJFOU
    4FDSFUͳͲΛ҆શʹѻ͍͍ͨ
25. 
    
26. ύϥϝʔλετΞʹઃఆ஋Λ֨ೲ
     aws ssm put-parameter \ --name /ClientId \ --value CLIENT_ID_XXXX \ --type String
27. λεΫఆ͔ٛΒࢀর
     ContainerDefinitions: - Name: app ... Secrets: - Name: CLIENT_ID ValueFrom: !Sub "arn:aws:ssm:ap-northeast-1:${AWS::AccountId}:parameter/ClientId" - Name: CLIENT_SECRET ValueFrom: !Sub "arn:aws:ssm:ap-northeast-1:${AWS::AccountId}:parameter/ClientSecret" λεΫ಺ͷ؀ڥม਺Ͱ஋ΛऔಘͰ͖Δ
28. 
     ΍ͬͯΈͨ
29. 
     wର৅σʔλສ݅
    w424΁ͷσʔλૹ৴ʹ࣌ؒ
    wʢͳΜ͔վળ͍ͨ͠ؾ͕͢Δ
    w&$4ͷόονॲཧ͕࣌ؒະຬͰऴྃ
30. 
     w4%,Ͱ4XJUDI
    3PMF͢Δʹ͸ͻͱखؒඞཁ
    w424͸࢓૊ΈΛཧղ͔ͯͭ͑͠͹ɺ͘͢͝ศ ར
    w&$4ͷฒྻλεΫΛ࢖ͬͯɺεέʔϥϒϧʹ͠ Α͏