Kubernetesの上に作る、統一されたマイクロサービス運用体験

SRE NEXT 2022 5/14(土) Room B 14:15 ~ 14:35 の発表資料です。

以下資料中で参照しているURL一覧です。

- https://cloud.google.com/config-connector/docs/overview
- https://cloud.google.com/blog/ja/products/containers-kubernetes/understanding-configuration-as-data-in-kubernetes
- https://cloud.google.com/spanner
- https://learning.oreilly.com/library/view/managing-kubernetes/9781492033905/ch03.html#fig0301
- https://github.com/mercari/spanner-autoscaler
- https://github.com/cloudspannerecosystem/autoscaler
- https://cloud.google.com/architecture/autoscaling-cloud-spanner
- https://raw.githubusercontent.com/cloudspannerecosystem/autoscaler/master/resources/architecture-per-project.png
- https://cloud.google.com/spanner/docs/backup
- https://speakerdeck.com/keisukeyamashita/spinnakerdeshi-jian-surumaikurosabisufalse-an-quan-naririsuhurotobesutopurakuteisu?slide=57

Transcript

1. 
   ,VCFSOFUFTͷ্ʹ࡞Δɺ
   ౷Ұ͞ΕͨϚΠΫϩαʔϏεӡ༻ମݧ 43&
   /&95
   
    4BU
   3PPN
   #
   !ULVDIJLJ
2. w!ULVDIJLJ
   wϝϧϖΠ43&
   wTQBOOFSBVUPTDBMFSͷϝϯ ςφ
   w(PQIFS
   8)0".* 
3. "HFOEB wϝϧϖΠͷϚΠΫϩαʔϏεͷӡ༻
   w,VCFSOFUFTΧελϜίϯτϩʔϥʔΛ׆༻ͨ͠$MPVE
   4QBOOFSͷΦʔτεέʔϧ
   w43&ϚωʔδυίϯςφΠϝʔδΛ࢖༻ͨ͠$MPVE
   4QBOOFSͷόοΫΞοϓ
   wӡ༻ͷ࢓૊ΈΛ,VCFSOFUFT্ʹ౷Ұ͢ΔϝϦοτ
   w·ͱΊ
4. ϝϧϖΠͷϚΠΫϩαʔϏεͷӡ༻
5. wϚΠΫϩαʔϏεͰ͸εέʔϥϒϧͳ૊৫Λͭ͘ΔͨΊʹ֤νʔ ϜͰ։ൃ͔Βӡ༻·Ͱߦ͏
   wνʔϜ͝ͱʹӡ༻Λ೚ͤͯ͹Β͹Βͳ࢓૊ΈΛ࡞ͬͯ͠·͏͜ͱͰ ൃੜ͢Δ՝୊
   wπʔϧ։ൃίετͷ૿େ
   wڞ௨ͷӡ༻ϧʔϧΛద༻Ͱ͖ͳ͍
   w43&͕ϑΥϩʔͰ͖ͳ͍
   wڞ௨ͷϓϥοτϑΥʔϜ্ͰϚΠΫϩαʔϏεΛӡ༻͢Δ͜ͱͰղܾ ϚΠΫϩαʔϏεӡ༻ʹ͓͚Δ՝୊
6. ϝϧϖΠͷαʔϏεن໛ .JDSPTFSWJDFT ,VCFSOFUFT
   1PET   
7. ϝϧϖΠͷϚΠΫϩαʔϏεج൫ w։ൃऀ͕ΦʔφʔγοϓΛ࣋ͬͯαʔϏεΛߏஙɾӡ ༻͢ΔͨΊͷϓϥοτϑΥʔϜ
   wϝϧΧϦͷ.JDSPTFSWJDFT
   1MBUGPSN
   5FBN͕ߏஙɾӡ༻
   w5FSSBGPSNͰ($1ϦιʔεΛ؅ཧ *B$
   w,VCFSOFUFT্ͷϦιʔεΛ:".-Ͱ؅ཧ $B%
   w$*ܦ༝ͰͷLVCFDUM
   BQQMZ
   w4QJOOBLFSͰσϓϩΠ *B$
   
   *OGSBTUSVDUVSF
   BT
   $PEF
   $B%
   
   $PO fi HVSBUJPO
   BT
   %BUB
8. $PO fi HVSBUJPO
   BT
   %BUB wΠϯϑϥΛ͋Δ΂͖ঢ়ଶΛએݴͨ͠σʔλͱ ͯ͠؅ཧ
   w,VCFSOFUFTͷ:".-ͳͲ
   w($1ͷϦιʔεΛ$B%Ͱ؅ཧ͢Δ,VCFSOFUFT ίϯτϩʔϥʔ΋͋Δ
   w$PO fi H
   $POOFDUPS IUUQTDMPVEHPPHMFDPNCMPHKBQSPEVDUTDPOUBJOFSTLVCFSOFUFTVOEFSTUBOEJOHDPO fi HVSBUJPOBTEBUBJOLVCFSOFUFT
9. ϝϧϖΠ43&ͷ໾ׂ wϚΠΫϩαʔϏεϓϥοτϑΥʔϜ্Ͱͷ ӡ༻αϙʔτ
   wϚΠΫϩαʔϏεڞ௨ͷΠϯϑϥͷӡ༻
   wӡ༻ʹ΋$B%ͷΑ͏ͳ࢓૊ΈΛఏڙ
   wຊηογϣϯͰ͸ͭͷࣄྫΛ঺հ $B%
   
   $PO fi HVSBUJPO
   BT
   %BUB
10. ,VCFSOFUFTΧελϜίϯτϩʔϥʔ
    Λ׆༻ͨ͠$MPVE
    4QBOOFSͷ
    Φʔτεέʔϧ
11. $MPVE
    4QBOOFS w($1ͷϑϧϚωʔδυϦϨʔγϣφϧɾ σʔλϕʔε
    wμ΢ϯλΠϜͳ͠Ͱ/PEFΛ૿ݮՄೳ
    wϝϧϖΠͷඪ४తͳσʔλϕʔεͱͯ͠ར༻ IUUQTDMPVEHPPHMFDPNTQBOOFS
12. /PEFΛ૿ݮͤ͞Δӡ༻ϑϩʔ %FWFMPQFS 1VMM
    3FRVFTU $MPVE
    4QBOOFS 
    /PEFT UFSSBGPSN
    BQQMZ
    PO
    $*
13. /PEFΛ૿ݮͤ͞Δӡ༻ϑϩʔ %FWFMPQFS 1VMM
    3FRVFTU $MPVE
    4QBOOFS 
    /PEFT UFSSBGPSN
    BQQMZ
    PO
    $* 8BJU
    GPS
    $* 8BJU
    GPS
    $* 5BLF
    B
    MPOH
    UJNF
14. $MPVE
    4QBOOFSͷӡ༻ʹ͓͚Δ՝୊ w$MPVE
    4QBOOFSͷ$16࢖༻཰ߴಅʹΑΓϨΠςϯγ ͕૿Ճͯ͠΋/PEF௥Ճʹ͕͔͔࣌ؒΔ
    wͲͷ͘Β͍/PEF௥Ճ͢Δඞཁ͕͋Δͷ͔൑அ
    w1VMM
    3FRVFTUͷ࡞੒
    wίʔυϨϏϡʔͱ$*଴ͪ $16࢖༻཰ʹԠͯ͡Φʔτεέʔϧ͍ͤͨ͞
15. Φʔτεέʔϧͷݕ౼ w($1ϓϩμΫτΛ૊Έ߹ΘͤΔ
    w$MPVE
    4DIFEVMFS
    w$MPVE
    'VODUJPOT
    w$MPVE
    .POJUPSJOH
16. Φʔτεέʔϧͷݕ౼ w͜ͷ৔߹ɺͭͷӡ༻ํ๏͕ߟ͑ΒΕΔ
    w֤ϚΠΫϩαʔϏεͰӡ༻
    w($1ϓϩδΣΫτ͝ͱʹՔಇ͢Δ
    w($1ϓϩδΣΫτͰՔಇ͠ɺෳ਺ͷ($1ϓϩ δΣΫτͷ4QBOOFSΛ؅ཧ͢Δ
    w தԝूݖతʹ43&͕ӡ༻
17. Φʔτεέʔϧͷݕ౼ w($1ϓϩμΫτͷ૊Έ߹ΘͤͰ΋΍Γ͍ͨ͜ͱ͸࣮ ݱͰ͖Δ͕ʜ
    w֤ϚΠΫϩαʔϏεͰӡ༻
    w։ൃऀ͕ӡ༻͢Δ΋ͷ͕૿͑Δ
    wதԝूݖతͳӡ༻
    w43&ͷෛ୲͕େ͖͍
    w։ൃऀͷΦʔφʔγοϓΛଛͳ͏
18. Φʔτεέʔϧͷݕ౼ w,VCFSOFUFTϦιʔεͱͯ͠؅ཧͰ͖Ε͹ طଘͷӡ༻ϑϩʔʹ৐ͤΒΕΔ
    wϦιʔεͷϨϏϡʔɾσϓϩΠΛϚΠΫϩ αʔϏεͷνʔϜ಺Ͱ׬݁Ͱ͖Δ ,VCFSOFUFTΧελϜίϯτϩʔϥʔΛ࣮૷
19. ,VCFSOFUFTΧελϜίϯτϩʔϥʔ wϢʔβ͕ಠࣗʹఆٛͨ͠ϦιʔεΧελϜ Ϧιʔεʹର͢ΔॲཧΛߦ͏ίϯτϩʔϥʔ
    w3FDPODJMJBUJPO
    MPPQʹΑΓɺએݴͨ͠ঢ়ଶ ͱ࣮ࡍͷঢ়ଶΛൺֱ͠ɺ͋Δ΂͖ঢ়ଶʹಉ ظ͠ଓ͚Δ
20. 3FDPODJMJBUJPO
    MPPQ IUUQTMFBSOJOHPSFJMMZDPNMJCSBSZWJFXNBOBHJOHLVCFSOFUFTDIIUNM fi H
21. 3FDPODJMJBUJPO
    MPPQ wݱࡏͷ/PEF਺Λ͋Δ΂͖/PEF਺ʹಉظ ͢Δɺͱߟ͑Δͱ૬ੑ͕ྑͦ͞͏
    w)1"ʹࣅ͍ͯΔ )1"
    
    )PSJ[POUBM
    1PE
    "VUPTDBMFS
22. TQBOOFSBVUPTDBMFS wIUUQTHJUIVCDPNNFSDBSJTQBOOFS BVUPTDBMFS
    w)1"ͷΑ͏ʹ$MPVE
    4QBOOFSͷ/PEFΛΦʔτ εέʔϧ͢Δ,VCFSOFUFTΧελϜίϯτϩʔ ϥʔ
    w೥݄ϦϦʔε )1"
    
    )PSJ[POUBM
    1PE
    "VUPTDBMFS
23. TQBOOFSBVUPTDBMFSΛ׆༻ͨ͠ӡ༻ 43& %FWFMPQFS .BOBHF ,VCFSOFUFTΧελϜϦιʔε 4QBOOFS"VUPTDBMFS
    ΧελϜϦιʔε LVCFDUM
    BQQMZ TQBOOFSBVUPTDBMFS $MPVE
    4QBOOFS 4DBMFVQEPXO "VUPTDBMF
    DPO fi HVSBUJPO 'PDVT
    PO
    DPOUSPMMFS
    NBOBHFNFOU 0OMZ
    NBOBHF
    :".-
24. ͜͜·Ͱͷ·ͱΊ w$MPVE
    4QBOOFSͷ/PEFΛ૿ݮͤ͞Δӡ༻ϑ ϩʔʹ՝୊͕͋ΓΦʔτεέʔϧͷඞཁੑΛ࣮ײ
    w,VCFSOFUFTΧελϜίϯτϩʔϥʔͷ 3FDPODJMJBUJPO
    MPPQͷ࢓૊Έʹண໨
    w)1"ͷΑ͏ʹ$MPVE
    4QBOOFSΛΦʔτεέʔϧ ͢Δ,VCFSOFUFTΧελϜίϯτϩʔϥʔΛ࣮૷
25. DMPVETQBOOFSFDPTZTUFNBVUPTDBMFS wIUUQTHJUIVCDPN DMPVETQBOOFSFDPTZTUFNBVUPTDBMFS
    w($1ϓϩμΫτΛ૊Έ߹ΘͤͯΦʔτεέʔϧ
    w೥݄ࠒϦϦʔε
    w$16λεΫ༏ઌ౓ߴҎ֎ͷϝτϦΫεͰεέʔ ϦϯάՄೳ https://cloud.google.com/architecture/autoscaling-cloud-spanner
26. DMPVETQBOOFSFDPTZTUFNBVUPTDBMFS https://raw.githubusercontent.com/cloudspannerecosystem/autoscaler/master/resources/architecture-per-project.png
27. DMPVETQBOOFSFDPTZTUFNBVUPTDBMFS wҎԼͷཧ༝ʹΑΓ࠾༻ͤͣ
    w($1ϓϩμΫτΛ૊Έ߹ΘͤΔ࣮૷Ͱڍ͛ͨ ໰୊͕ಉ༷ʹൃੜͦ͠͏
    w,VCFSOFUFTϦιʔεͱͯ͠Φʔτεέʔϧ ͷઃఆΛ؅ཧͰ͖Δํ͕ӡ༻ମݧ͕ྑ͍
28. 43&ϚωʔδυίϯςφΠϝʔδΛ
    ར༻ͨ͠$MPVE
    4QBOOFSͷ
    όοΫΞοϓ
29. $MPVE
    4QBOOFSͷόοΫΞοϓ w$MPVE
    4QBOOFSʹόοΫΞοϓΛࣗಈͰఆظతʹ࣮ߦ͢Δ ػೳ͸ͳ͍
    w$MPVE
    4DIFEVMFS͔Β"1*ϦΫΤετΛૹΔ͜ͱͰఆظతʹ ࣮ߦՄೳ
    wͭͷόοΫΞοϓํ๏ ύϑΥʔϚϯε
    Өڹ ϙʔλϏϦςΟ อ࣋ظؒ όοΫΞοϓ ͳ͠ Πϯελϯεʹ
    ඥ͖ͮҠಈෆՄ ࠷௕೥ؒ ΤΫεϙʔτ ௿ ೚ҙͷετϨʔδ্ Ͱอ؅ ࡟আ͢Δ·Ͱ https://cloud.google.com/spanner/docs/backup
30. $MPVE
    4QBOOFSͷόοΫΞοϓ w$MPVE
    4DIFEVMFS୯ମͰ΋όοΫΞοϓͷऔ ಘ͸Ͱ͖Δ͕
    wΤΫεϙʔτδϣϒࣦഊ࣌ʹ4MBDL௨஌͍ͨ͠
    wΤΫεϙʔτδϣϒͷ࣮ߦ࣌ؒΛܭଌ͍ͨ͠ πʔϧͷ࣮૷Λݕ౼
31. πʔϧΛ࣮૷͢Δͱ͖ͷબ୒ࢶ w"QQ
    &OHJOF
    $SPOαʔϏε
    w$MPVE
    4DIFEVMFS
    
    $MPVE
    'VODUJPOT
    w$MPVE
    4DIFEVMFS
    
    $MPVE
    3VO
32. ӡ༻ޙʹൃੜͨ͠໰୊ w։ൃऀ͸"QQ
    &OHJOFΛૢ࡞͢ΔݖݶΛ࣋ ͨͳ͍ͷͰ43&͕ઃఆ
    wઃఆΛߋ৽͍ͨ͠৔߹΋43&ʹґཔ
33. 43& ӡ༻ޙʹൃੜͨ͠໰୊ "QQ
    &OHJOF "QQ
    &OHJOF 
    
     %FWFMPQFS %FQMPZ 
    
     #PUUMFOFDL )FBWZ
    XPSLMPBE 3FRVFTU 
    
    
34. ӡ༻ޙʹൃੜͨ͠໰୊ͷվળ w"QQ
    &OHJOFͷσϓϩΠύΠϓϥΠϯΛߏங ͢Δ͜ͱͰ໰୊ΛղܾͰ͖ͦ͏Ͱ͸͋Δ͕
    wόοΫΞοϓͷͨΊ͚ͩʹ࢓૊ΈΛ௥Ճ͢Δ ͷ͸େม
    w؅ཧ͢Δ΋ͷ͕෼ࢄ͢Δ
35. ӡ༻ޙʹൃੜͨ͠໰୊ͷվળ w4QJOOBLFS
    1JQFMJOFΛ:".-Ͱ࡞੒͢Δ࢓૊ Έ͕͋Δ
    w,VCFSOFUFT
    $SPO+PCͰ࣮ߦ͢Ε͹ɺ1JQFMJOF ͷߏஙͱ+PCͷઃఆΛ:".-Ͱ׬݁Ͱ͖Δ
    wΞϓϦέʔγϣϯͱಉ͡Α͏ʹ؅ཧͰ͖Δ IUUQTTQFBLFSEFDLDPNLFJTVLFZBNBTIJUBTQJOOBLFSEFTIJKJBOTVSVNBJLVSPTBCJTVGBMTFBORVBOOBSJSJTVIVSPUPCFTVUPQVSBLVUFJTV TMJEF
36. ӡ༻ޙʹൃੜͨ͠໰୊ͷվળ w,VCFSOFUFT
    $SPO+PCͰ࣮ߦ͢ΔͨΊʹ͸ ίϯςφΠϝʔδ͕ඞཁ
    wϚΠΫϩαʔϏε͝ͱʹΠϝʔδΛ࡞੒
    wઐ༻ͷ%PDLFS fi MFΛ࡞੒ͯ͠ΠϝʔδΛϏϧυ
    w43&͕࡞੒ͨ͠ΠϝʔδΛ֤ϚΠΫϩαʔϏ εͰ࢖༻
37. 43&ϚωʔδυͳΠϝʔδΛ࢖༻ͨ͠ӡ༻ $POUBJOFS
    3FHJTUSZ %FWFMPQFS .BOBHF ,VCFSOFUFT
    $SPO+PC 3VO
    $%
    1JQFMJOF %FQMPZ 6TF
    *NBHF 43& :".- 'FUDI 'PDVT
    PO
    JNBHF
    NBOBHFNFOU 0OMZ
    NBOBHF
    :".-
38. ͜͜·Ͱͷ·ͱΊ w$MPVE
    4QBOOFSͷόοΫΞοϓΛ։ൃऀ͕ ΦʔφʔγοϓΛ࣋ͬͯઃఆͰ͖ͳ͔ͬͨ
    w43&ͷෛՙ΋ߴ͔ͬͨ
    w։ൃऀࣗ਎ͰઃఆͰ͖ΔΑ͏ʹ؀ڥΛ੔උ
    w։ൃऀͷΦʔφʔγοϓΛଛͳΘͣɺ43&ͷෛ ՙΛܰݮ
39. ӡ༻ͷ࢓૊ΈΛ,VCFSOFUFT্ʹ
    ౷Ұ͢ΔϝϦοτ
40. ࢓૊ΈΛ,VCFSOFUFT্ʹ౷Ұ͢ΔϝϦοτ w:".-Λॻ͍ͯϦιʔεΛσϓϩΠ͢Δͩ ͚ͱ͍͏खܰ͞
    w։ൃऀ͕؅ཧ͢Δͷ͸:".-͚ͩ
    wπʔϧ͝ͱʹΩϟονΞοϓ͢΂͖͜ͱ͕গ ͳ͍ͷͰӡ༻ͷෛ୲͕ݮΔ
41. ࢓૊ΈΛ,VCFSOFUFT্ʹ౷Ұ͢ΔϝϦοτ wઃఆ࡞ۀ͕νʔϜ಺Ͱ׬݁͢ΔͨΊΦʔ φʔγοϓΛଛͳΘͳ͍
    wӡ༻ͷ࢓૊ΈΛઃܭ͢Δ্Ͱॏཁ
    wεέʔϥϒϧͳ૊৫Λͭ͘ΔͨΊʹ͸ඞཁ ෆՄܽ
42. ·ͱΊ
43. ຊηογϣϯͷ·ͱΊ wӡ༻ମݧ޲্ͷͨΊʹӡ༻ʹ$B%ͷ࢓૊ΈΛ औΓೖΕͨࣄྫΛ঺հ
    wӡ༻ͷ࢓૊ΈΛ,VCFSOFUFT্ʹ౷Ұ͢Δ͜ͱ ͰҎԼΛ࣮ݱ
    w:".-Λॻ͍ͯσϓϩΠ͢Δ͚ͩͷखܰ͞
    w։ൃऀͷΦʔφʔγοϓΛଛͳΘͳ͍ӡ༻؀ڥ $B%
    
    $PO fi HVSBUJPO
    BT
    %BUB