Share
Embed
Transcript
53 slides
-
2 Platform Engineering Division SRE team Engineering Manager ΤϯδχΞϦϯάϚωʔδϟʔͱͯ͠Fintechࣄ
ۀ͓ΑͼϝϧΧϦͷSREνʔϜΛ୲͠ɺ৫ ԣஅͰ৴པੑ্ʹऔத • SLOΛ༻͍ͨࢹج൫ͷߏங • ΠϯγσϯτϚωδϝϯτମ੍ͷڧԽ • BCP/DRઓུͷࡦఆɾਪਐ foostan -
12 αʔϏε֦େʹ͏ • ෳࡶԽ͢ΔγεςϜ • σʔλͷ૿Ճ • ։ൃ৫ͷ֦ுɺνʔϜͷ૿Ճ • σϓϩΠසͷ૿Ճ
• ϝϯςφϯείετͷ૿Ճ • Πϯγσϯτͷ૿Ճ • ੜ࢈ੑͷԼ • etc 2013 ~ 2016 ϞϊϦε / LAMPͷݶք -
13 2017 ϚΠΫϩαʔϏεͷஈ֊తͳҠߦ ϚΠΫϩαʔϏε + Google Cloud • ϞϊϦε͔Βঃʑʹ •
ϚΠΫϩαʔϏε୯ҐͰ։ൃ/ӡ༻ • API GatewayʹΑΔΠϯλʔ ϑΣʔεͷ౷Ұ • etc ৫ͷٸܹͳ֦େʹ͏αʔϏε։ൃ ͷࠔӡ༻ͷ֦ுੑͷϘτϧωοΫ ղফΛਤΔ ϩΰग़యɿGoogle Cloud ϓϩμΫτ ΞΠίϯ -
15 ϝϧίΠϯͷϦϦʔεʹ͍ɺ Platformӡ༻มԽ ҉߸ࢿ࢈ަۀΛਐΊΔͨΊ ͷηΩϡϦςΟରࡦίϯϓϥ ΠΞϯεཧͷͨΊʹ͋ΒΏΔ ͷͷ͕ٻΊΒΕͨ Mercoin Development Tool
Production Mercari Development Tool Production 2023 ϝϧίΠϯϩʔϯνͱϓϥοτϑΥʔϜ ϩΰग़య: Google Cloud ϓϩμΫτ ΞΠίϯ GitHub Logos and Usage https://brandfetch.com/okta.com -
16 ϝϧίΠϯͱϝϧΧϦؒ Private Service Connect Ͱଓ͞ Ε ϚΫϩαʔϏεͷґଘؔ࠷খݶʹͳΔΑ͏ઃܭ͞Εͨ 2023 ϝϧίΠϯͷϩʔϯνͱΫϥελ
ϩΰग़య: Google Cloud ϓϩμΫτ ΞΠίϯ https://brandfetch.com/fastly.com -
17 ϝϧϖΠͷϨΠϠʔߏԽ͞ΕϚ ΠΫϩαʔϏε͕ഁ࢝͠ΊΔ API Microservices ϨΠϠʔʹศ རAPI͕ੜ·ΕΔ • ґଘ͕ूத •
ϝϯςίετͷ૿େ • ᐆດͳڥք • SPOFԽ • etc 2024 Re Architecture ϩΰग़యɿGoogle Cloud ϓϩμΫτ ΞΠίϯ -
18 2024 ~ Re Architecture | ٯίϯΣΠͷ๏ଇʹΑΔղܾ ॎׂΓͷ৫ߏͷมߋ • API
MicroservicesϨΠϠʔͷཧ • ґଘ͕ूத͍ͯͨ͠APIͷղମ͕ ਐߦ gRPC Federation • ϝϧΧϦൃͷOSS • Protocol Buffersͷఆٛʹैͬͯ BFFαʔϏεͱΠϯλʔϑΣʔε Λੜ͢Δ ϩΰग़యɿGoogle Cloud ϓϩμΫτ ΞΠίϯ -
19 ϚΠΫϩαʔϏεઓུͷӡ༻ Ͱग़͖ͯͨ • ଟछଟ༷ͳϚΠΫϩαʔϏε • ϥΠϒϥϦίϯϙʔωϯτͳ ͲΛৗʹ࠷৽ʹอͭͷࠔ • ৽نߏங͢ΔͨΊͷπʔϧε
ΫϦϓτɺνΣοΫϦετͳͲ ͷΤίγεςϜॆ࣮͍ͯ͠Δ ͕ɺֶशίετ͕ߴ͍ • etc 2025 ϚΠΫϩαʔϏεͷۤ -
20 ࣾͰϞϊϨϙϞδϡϥʔ ϞϊϦεͷߏ͕ग़࢝Ίͨ άϩʔόϧΞϓϦͰͷ࠾༻ • طଘͷΤίγεςϜΛ׆ ༻ͭͭ͠ɺӡ༻ͷॊೈੑ Λҡ࣋͢Δ •
ॳظ։ൃͷεϐʔυͱক དྷతͳεέʔϥϏϦςΟ ͷόϥϯεΛऔΔ 2025 ϞδϡϥʔϞϊϦε ࢀߟ: άϩʔόϧల։Λࢧ͑Δج൫ͷཪଆ https://engineering.mercari.com/blog/entry/20251007-behind-the-infrastructure-powering-global-expansion/ [mercari GEARS 2025] Building Foundation for Mercari’s Global Expansion https://speakerdeck.com/mercari/mercari-gears-2025-building-foundation-for-mercaris-global-expansion -
24 ໌֬ͳൣғͱᐆດͳൣғ ໌֬ͳൣғ • ࣗͷαʔϏε໌֬ͰͦͷSLO Λཧ͢Δͷඞਢ • ൣғ֎ͷଞͷSLOΛؾʹ͢Δʹ ݶք͕͋Δ ᐆດͳൣғ
• ڞ௨APIͷ୲Ͳ͔͜ • ଞϚΠΫϩαʔϏεʹґଘͯ͠Ξ ϥʔτ͕τϦΨʔ͞ΕΔ͜ͱ͠ ͠ ϩΰग़యɿGoogle Cloud ϓϩμΫτ ΞΠίϯ -
25 ϚΠΫϩαʔϏεຖͷSLOཧ ͯ͢ͷΤϯυϙΠϯτʹରͯ͠ཏ తʹSLOΛઃఆ • SLOϕʔεͷϞχλʔઃஔ • ৄࡉʹෆ۩߹ՕॴΛݕ͠ɺΞ ϥʔτΛτϦΨʔՄೳ •
ϚΠΫϩαʔϏε୯ମͰ100Λ ͑ΔSLOΛཧ • SLOͷৼΓฦΓΞοϓσʔτ͕ ࠔʹ • ϚΠΫϩαʔϏεؒͰSLOͷෆ ߹͕ൃੜ ϩΰग़యɿGoogle Cloud ϓϩμΫτ ΞΠίϯ -
26 োൣғͷಛఆࠔ ͲͷAPI͕Ͳ͜ͰϦΫΤετ͞ΕΔ ͔ͷѲࠔ • Ξϥʔτ͕τϦΨʔ͞Εͯ ɺ͓٬͞·Өڹͱ݁ͼ ͔ͭͳ͍ • ൣғ֎ͷνʔϜʹͱͬͯԿ
͕ى͖͍ͯΔ͔Ѳ͢Δͷ ͍͠ • ݁Ռతʹো෮چ͕ΕΔ͜ ͱʹͳΔ ϩΰग़య: Google Cloud ϓϩμΫτ ΞΠίϯ Cloudflare Press Kit -
27 Critical User JourneyʢCUJʣͷಋೖ ͓٬͞·ʹϑΥʔΧε • Ұ࿈ͷ͓٬͞·ମݧʹ ରͯ͠SLOΛఆٛ • ͓٬͞·Өڹ͕໌֬ʹ
ͳΓɺΠϯγσϯτͷ શମѲ͕Մೳʹ • ݱ࣮తͳͷCUJΛఆ ٛ࣋͠ଓతͳཧ͕Մ ೳʹ ϩΰग़యɿGoogle Cloud ϓϩμΫτ ΞΠίϯ -
28 CUJϕʔεSLOͷ࣮ 40ͷCUJΛఆٛ • ྫʣ ◦ ݕࡧ ◦ ग़ɾߪೖɾൃૹ •
ը໘ૢ࡞ + ը໘ભҠ • Availableͳঢ়ଶΛఆٛ • ίΞػೳʹϑΥʔΧε ࢀߟ: [SRE NEXT] ෳࡶͳγεςϜʹ͓͚ΔUser Journey SLOͷಋೖ https://speakerdeck.com/yakenji/sre-next-fu-za-nasisutemuniokeruuser-journey-slonodao-ru -
29 CUJϕʔεSLOͷ࣮ طଘͷςετϑϨʔϜϫʔΫΛ ྲྀ༻ͯ͠୳ࡧΛࣗಈԽ • ΞϓϦʑมԽ͢Δ → ΫϦςΟΧϧAPIมԽ ͢Δ •
୳ࡧΛࣗಈԽͯ͠ৗʹ࠷৽ ͷঢ়ଶΛҡ࣋͢Δඞཁ͕͋ Δ -
30 CUJϕʔεSLOͷ࣮ LookerͰ݁ՌͷՄࢹԽ +TerraformͰϞχλϦϯάө • ΫϦςΟΧϧAPIͷՄࢹԽͱ APMͷөࣗಈԽ • Λݮͭͭ͠౷ҰతͳϞ χλϦϯάΛ࣮ࢪ
ϩΰग़య: Google Cloud ϓϩμΫτ ΞΠίϯ HashiCorp Brand Guidelines -
32 Datadog WorkflowsʹΑΔݪҼಛఆ CUJϕʔεSLOΛτϦΨʔʹ༷ͯ͠ʑͳΞΫ γϣϯʹͭͳ͛Δ • ্ҐϨΠϠʔ(API Gateway)ͷϝτϦΫ εΛར༻͍ͯ͠ΔͨΊͲ͜Ͱ͕ൃੜ ͔ͨ͠ಛఆͰ͖ͳ͍
• Datadog WorkflowsΛར༻ͯ͠ɺಈతʹ ใΛऔಘͯ͠ݪҼΛಛఆ͠୲νʔϜ Λݺͼग़͢ • ࠓޙαʔϏεͷεϖοΫใ PlaybookΛར༻͠ɺࠜຊݪҼੳࢭ ݂ରԠͷࣗಈԽʹͭͳ͛Δ ϩΰग़య: Google Cloud ϓϩμΫτ ΞΠίϯ Cloudflare Press Kit https://slack.com/intl/ja-jp/media-kit https://brandguides.brandfolder.com/pagerduty/logo -
35 ৴པੑΛ༳Δ͕͢ϦεΫ ϦεΫ • Րࡂɺେ(टԼܕ) • େنγεςϜো • αΠόʔ߈ܸ Өڹൣғ
• κʔϯن • Ϧʔδϣϯن • ґଘ͢ΔαʔϏε / ύʔτφʔاۀ -
36 ϦεΫʹର͢Δඋ͑ BCP/DR Business Continuity Plan = ࣄۀܧଓܭը • ࡂγεςϜোͳͲͷ༗ࣄͷࡍʹɺ͓٬͞·ͷӨڹΛ࠷খ
ݶʹ͑ɺॏཁۀΛૣظʹ෮چɾܧଓͤ͞Δ͜ͱͰɺࣾձతͳ ΛՌͨ͢ • γεςϜʹݶΒͣɺਓݐ/ઃඋؚ·ΕΔ Disaster Recovery • ॏཁۀΛఆΊͨ෮چඪ࣌ؒʢRTOʣͱඪ෮چ࣌ʢRPOʣ ʹج͖ͮɺใγεςϜ͓ΑͼۀϓϩηεΛਝʹ෮چͤ͞Δ • BCPʹؚ·ΕΔɺ۩ମతͳγεςϜ෮چʹؔ͢Δ͜ͱ -
37 ԿΛकΔ͔ ͯ͢ͷαʔϏε/ػೳΛ100%Քಇ͠ଓ͚Δ͜ͱͰ͖ͳ͍ • ϏδωεӨڹੳ(BIA)Λߦ͍ॏཁۀΛఆΊΔ • ॏཁۀ͕ఀࢭͨ͠ͱ͖ͷϦεΫԿ͔ ◦ ๏ྩҧ ◦
ϒϥϯυՁͷԼܦࡁతଛࣦ ◦ αʔϏεʹΑͬͯҟͳΔ͔ ▪ ϝϧΧϦɺϝϧϖΠɺϝϧίΠϯɺetc • ୡ͖͢෮چඪΛఆΊͯϦεΫͱίετͷόϥϯεΛߟ͑Δඞཁ͕ ͋Δ -
38 ෮چඪ • RLO(Recovery Level Objective): ۓٸͷ෮چର = ॏཁۀ •
RTO(Recovery Time Objective) : ͍ͭ·Ͱʹ෮چͤ͞Δ͔ • RPO(Recovery Point Objective): Ͳͷ࣌·ͰͷσʔλΛ෮چͤ͞Δ͔ -
41 Multi-RegionߏͷҙɿSpannerͷ߹ ಛ • ߴ͍Մ༻ੑͱSLA • RPO: 0 ͷ࣮ݱ ίετ
• 3~4ഒ্ঢ͢Δ(ಡΈॻ͖ྔɺσʔλྔʹΑͬͯҟͳΔ) ωοτϫʔΫԆʹΑΔੑೳྼԽ • ॻ͖ࠐΈ1 commit ͋ͨΓms~ेmsͷϨΠςϯγѱԽ͕ఆ͞ΕΔ • εϧʔϓοτͷ্ݶ͕Լ͕Δ • ΞϓϦέʔγϣϯଆͰͷରԠ͕ඞཁʹͳΔ߹͕͋Δ -
42 Multi-RegionߏͷҙɿGCSͷ߹ σʔλಉظͷԆ • 1࣌ؒͰ99.9%͕ಉظ͞ΕɺΓ 12࣌ؒҎΛඪͱ͍ͯ͠Δ (RPO: 12࣌ؒ) • ඞཁʹԠͯ͡Dual-regionͷλʔ
ϘϨϓϦέʔγϣϯ(RPO: 15) Λར༻͢Δඞཁ͕͋Δ • RPO: 0ͷ߹ͷ༻్ͱͯ͠ ͑ͳ͍ ϩΰग़య: Google Cloud ϓϩμΫτ ΞΠίϯ -
43 Multi-Region vs ࠷খߏ ίετཧతͳڑ͕Ε͍ͯΔ͜ͱͷ੍ɺγεςϜͷෳࡶ͞ʹΑΔཧίετΛߟ͑ ͯʮकΔ͖ͷʯͷݱ࣮ղΛ୳Δ ϩΰग़య: Google Cloud ϓϩμΫτ
ΞΠίϯ Cloudflare Press Kit HashiCorp Brand Guidelines -
44 ґଘ͢ΔαʔϏε / ύʔτφʔاۀͱͷBCPࠩҟ αʔϏεࣾͰ݁ͤͣʹ ύʔτφʔاۀͳͲ֎෦αʔϏ εʹґଘ͢Δ͜ͱ͕ଟ͍ ࣗͨͪͷBCPઓུΛ͑ͨ ͱͯ͠ɺ֎෦ʹґଘ͢Δ߹ ͦΕΒͷରԠߟ͓͔͑ͯ
ͳ͍ͱ͍͚ͳ͍ ϩΰग़య: Cloudflare Press Kit -
45 ґଘ͢ΔαʔϏε / ύʔτφʔاۀͱͷBCPࠩҟ RTO͕ಉਫ४Ͱ͋Εಉ͡λ ΠϛϯάͰ෮چ͕Մೳ ͔͠͠ଟ͘ͷ߹ɺ෮چඪ ҟͳΔͨΊɺ֎෦αʔϏε͕ ෮چ͠ͳ͍͜ͱΛߟྀ͢Δඞཁ
͕͋Δ ϩΰग़య: Cloudflare Press Kit -
48 ෮چ܇࿅ γεςϜͰकΕͳ͍෦܇࿅Ͱิ͏ • RPO܇࿅ͯ͠ߴΊΔ͜ͱͰ͖ͳ͍͕ɺ෮چ·Ͱ ͷ࣌ؒͷॖʹΑͬͯRTOΛߴΊΔ͜ͱՄೳ • ܇࿅ʹΑͬͯRPO/RTO͕ୡͰ͖Δ͔Ͳ͏͔ͷ֬ೝΛ ߦ͏ •
γεςϜৗʹมԽ͍ͯ͠ΔͷͰఆظతʹߦ͏ඞཁ͕ ͋Δ ͜ΕେنোࡂʹݶΒͣʑͷΠϯγσϯτʹؔ ͯ͠ಉ͡ -
49 ΠϯγσϯτϨεϙϯεͱֶͼ োࣦഊ͔ΒֶΜͰ࣍ʹ׆͔͢ͷ͕ॏཁ • ৼΓฦΓ ◦ ݪҼੳɺ߃ٱରԠ͓Αͼ࠶ൃࢭͷཱҊ ◦ γεςϜͷमਖ਼ɺΞʔΩςΫνϟͷมߋɺӡ༻ϓϩη εͷվળͳͲͰ࣍ʹܨ͛Δ
• Ϩϙʔτ࡞ / ڞ༗ ◦ ͚ࣾ / ࣾ֎͚ͷใࠂ ◦ φϨοδڞ༗ / ԣల։ • Πϯγσϯτੳ ◦ μογϡϘʔυ࡞ɺੳɺಛͷՄࢹԽ -
51 มԽ͢ΔڥͷதͰɺ৴པੑΛकΓଓ͚ΔͨΊʹ ΞʔΩςΫνϟมԽ͍ͯ͘͠ • ৽͍͠ϏδωεΛ࢝ΊΔͨΊ • ϑΣʔζͷมԽʹରԠ͢ΔͨΊ • ʮ·Δ͖ͷʯΛकΔͨΊ γεςϜյΕΔ
• ࡂΛࢭΊΔ͜ͱͰ͖ͳ͍ • ো͍ͭͰى͜Γ͏Δ -
52 มԽ͢ΔڥͷதͰɺ৴པੑΛकΓଓ͚ΔͨΊʹ ͓٬͞·ઢͷSLOΛ৴པੑͷࢦඪͱͯ͠ఆΊΔ • CUJϕʔεSLOͷΑ͏ʹ͓٬͞·ͷ৴པੑΛԽ͢Δ • ͓٬͞·Өڹ͕ग़ͨ߹ɺଈ࠲ʹݕ/ରԠͰ͖ΔΑ͏ʹ͢Δ कΔ͖ͷΛఆٛ͠ɺ͠ʹඋ͑Δ • ϦεΫͱίετΛߟྀͯ͠ΞʔΩςΫνϟΛ͢Δ
• ܇࿅ΠϯγσϯτϨεϙϯε͔Βͷֶͼॏཁ • ৼΓฦΓʹΑΔγεςϜͷमਖ਼ɺΞʔΩςΫνϟͷมߋɺӡ༻ϓϩηεͷ վળͳͲ͕৴པੑΛߴΊΔ