Upgrade to Pro — share decks privately, control downloads, hide ads and more …

入社1ヶ月でここまでやった!Findy Toolsインフラ支援の最適化

入社1ヶ月でここまでやった!Findy Toolsインフラ支援の最適化

https://findy.connpass.com/event/323099/
【増枠】Embedded SRE 現場に寄り添うアプローチの資料です。

Avatar for adachin0817

adachin0817

July 11, 2024
Tweet

More Decks by adachin0817

Other Decks in Programming

Transcript

  1. 3 ࣗݾ঺հ ҆ୡ ྋ(@adachin0817) ɾϑΝΠϯσΟ(ג) / ϓϩμΫτ։ൃ෦/Senior SRE ɾBlog: blog.adachin.me/wiki.adachin.me

    ɾTechBull(ٕज़ίϛϡχςΟ) techbull.cloud ɹɾʮSRE/ۦ͚ग़͠ΤϯδχΞͷϝϯλϦϯά ྦྷܭ300໊↑ʯ ɾ͔ͭͯ͸OSS൛VulsͷίϯτϦϏϡʔλʔ΍Πϕϯτओ࠵ͳͲ ɾ89೥ੜ·Εɺ౦ژ౎଍ཱ۠ग़਎Ͱ࡛ۄݝय़೔෦ࢢ͕஍ݩ ɾ࠺ͱϑϨϯνϒϧυοάͷࣂ͍ओͰ΋͋Δ
  2. Findy Toolsͱ͸ • ։ൃπʔϧʹಛԽͨ͠ϨϏϡʔαΠτ ◦ πʔϧબఆʹؔ͢Δෆ҆Λղফ ◦ ಋೖݕ౼ʹඞཁͳ৘ใΛఏڙ • ݱ৔ͰΑ͋͘Δ͓೰Έ

    ◦ ϝϦοτɾσϝϦοτ͕෼͔Βͳ͍ ◦ ৘ใऩूʹ͕͔͔࣌ؒΔ ◦ πʔϧબఆͷܾఆଧ ◦ ಋೖޙʹൃੜ͢Δ՝୊఺ • ͓͔͛͞·Ͱͨ͘͞Μͷʂ ◦ ձ༷ࣾʹࣥච͍͍͓ͨͩͯΓ·͢ʂ 10
  3. Findy Toolsͷٕज़ελοΫ • Ruby on Rails • Next.js • Docker

    • AWS ◦ ECS/Fargate ▪ Front Container ▪ Rails Container ▪ Nginx Container • ALB ◦ CloudFront ◦ Aurora/MySQL8 ◦ WAF • GitHub Actions(CI/CD) • Terraform(Module) • Datadog/Sentry 12
  4. Aurora όʔδϣϯΞοϓ • RDS SSL/TLSূ໌ॻͷߋ৽ ◦ ଞʹӨڹ͕ͳ͍Α͏ʹrds-ca-2019ͱಉ͡ΞϧΰϦζϜͷRSA2048Λ࢖༻͍ͯ͠Δ ◦ rds-ca-rsa2048-g1ʹมߋ ◦

    ࠶ىಈ΍μ΢ϯλΠϜ͸ͳ͍ͷͰαΫοͱ׬ྃ • Aurora MySQLϚΠφʔόʔδϣϯΞοϓ ◦ 8.0.mysql_aurora.3.04.0→8.0.mysql_aurora.3.07.0 ▪ μ΢ϯλΠϜ͋Δ͔໰୊ʹ͍ͭͯݒ೦ ▪ ·ͣ͸։ൃ؀ڥ(Docker)͔ΒόʔδϣϯΞοϓͱಈ࡞֬ೝ ◦ ͜ͷػձʹAmazon Aurora Blue/Green DeploymentsΛݕূ ▪ ͔͠͠1࣌ؒҎ্͔͔ΔͨΊɺStgͰͦͷ··όʔδϣϯΞοϓ(10෼) ◦ ࠶ىಈ΍μ΢ϯλΠϜ͸ͳ͔ͬͨͷͰɺ೔தʹରԠͯ͠ຊ൪׬ྃ 14 ʮೖࣾͯ͠Θ͔ͣ3೔ͰରԠʂʯ
  5. ؂ࢹ/ΦϒβʔόϏϦςΟ • ؂ࢹ/ΦϒβʔόϏϦςΟपΓ ◦ ࠓ·ͰSentryͰΤϥʔτϥοΩϯάͱAPMͷΈ͚ͩ ◦ Մ؍ଌੑେࣄͳͷͰ΍͍͖ͬͯ • ։ൃϝϯόʔ͸3໊ ◦

    ϝϯόʔͱग़ࣾ࣌ʹ1on1 ▪ ձ࿩͍ͯ͘͠தͰDatadogͰ౷Ұ͢Δ͜ͱʹͳͬͨ ▪ ΤϥʔτϥοΩϯά͸Sentryͷํ͕͍҆ͨΊܧଓ ▪ APM͸DatadogʹҠߦ • Πϯϑϥ/ΞϓϦέʔγϣϯϦιʔε ◦ ECS(CPU,Memory) ◦ ֎ܗ؂ࢹ(Response Time,Status Code) ◦ CloudFront(5xx,Error Rate) ◦ APM(Error,Latency) ◦ RDS(CPU,Memory,DeadLock,Network,Connection,Select Latency) ◦ Πϕϯτϩά(Error,Fail) ◦ Ξϥʔτ͸શͯTerraformԽɺμογϡϘʔυ͸खಈͰ؅ཧ 16
  6. SLI/SLO • SLI/SLOͷࡦఆ ◦ SLI/SLOͷӡ༻͸γϯϓϧͳߏ੒Ͱ࣮ݧ͍ͯ͘͠ ▪ τοϓϖʔδͷͦͷଞͷϖʔδͷද͕ࣔ࣌ؒ2ඵҎ಺Ͱ͋Δׂ߹͕99%Ҏ্ ▪ τοϓϖʔδ΍ͦͷଞͷϖʔδͷϦΫΤετॲཧ੒ޭ཰͕99%Ҏ্ •

    HTTPεςʔλείʔυ 4xxҎ֎ ▪ ΤϥʔόδΣοτ • 30೔ؒʹ0.1%ͷμ΢ϯλΠϜ͕100%ফඅ͞Εͨ৔߹ ▪ όʔϯϨʔτ • 30೔ؒʹ6͓࣌ؒΑͼաڈ30෼ؒͰ6ഒҎ্ͷ৔߹ ▪ আ֎ • ΠϯϑϥϦιʔε ◦ CPUɺMemoryɺωοτϫʔΫɺσΟεΫ ◦ සൟʹΞϥʔτ͕ग़ΔͨΊαʔϏεͷӨڹʹ͸ͳΒͳ͍ ▪ ৼΓฦΓ • ִिͰSLOͷݟ௚͠Λߦ͏ 19
  7. ։ൃ؀ڥͷվળ • ։ൃ؀ڥΛJustfileԽͯࣗ͠ಈߏங ◦ ։ൃ؀ڥ ▪ rails server ▪ Node.js(npm

    run) ▪ MySQL8(docker-compose) ◦ ࠓ·Ͱ͸खॱॻΛݟͳ͕Βߏங͍ͯͨ͠ ▪ खॱ͕ଟ͍ͨΊ࿙Ε͕ൃੜͯ͠͠·͏ ◦ Justfile(Task Runner Tool) ▪ https://github.com/casey/just ▪ Makefileͱൺֱ͢Δͱ • γϯϓϧͰ௚ײతͳߏจ • γΣϧεΫϦϓτʹࣅͯΔ • ґଘؔ܎ͷ؅ཧ͕؆୯ • ֶशίετ͕௿͍ 22
  8. ·ͱΊ/ࠓޙ΍Γ͍ͨ͜ͱ • ৴པੑͱจԽͮ͘Γ͸ձ࿩Ͱ͢ ◦ ີʹؔΘΓͳ͕Βࢧԉ͍ͯ͘͠ • Findy ToolsͰ࣮ݧతʹߦ͍ͬͯΔ؂ࢹपΓ ◦ ૊৫ԣஅʹΑΔϊ΢ϋ΢Λڞ༗

    ▪ Findy ▪ Findy Freelance ▪ Findy Team+ • AWSηΩϡϦςΟपΓ޲্ • AWSίετ࡟ݮ • ։ൃϝϯόʔ΁ͷੜ࢈ੑ޲্ ◦ ։ൃ؀ڥ ׬શDockerԽ ◦ ECS ArmҠߦ ◦ Stgෳ਺؀ڥ ࣗಈߏங ◦ σϓϩΠߴ଎Խ 25
  9. গ͠Ͱ΋ϑΝΠϯσΟͰಇ͘͜ͱʹ͝ڵຯ͋Ε͹ɺؾܰʹ͝࿈བྷ͍ͩ͘͞ʂ Ͳ͕͜దਖ਼͔Θ͔Βͳ͍ͱ͍͏ํ΋ؚΊͯɺΦʔϓϯϙδγϣϯ(ॻྨબߟ΍໘ஊ࣌ʹϙδγϣϯ͢Γ߹Θͤ)΍ΧδϡΞϧ໘ஊͷਃࠐΈΛߦ͍ͬͯ·͢ʂ 27 ※ืूঢ়گʹΑͬͯมಈ͠·͢ ։ൃνʔϜɾσβΠφʔ • CTOީิ • EM •

    ϑϩϯτΤϯυ • ػցֶशΤϯδχΞ • όοΫΤϯυ •σʔλΤϯδχΞ • SRE • UIσβΠφʔ Findy Career • ࣄۀ෦௕ީิ • ϚʔέςΟϯά • ࣄ຿Ξγελϯτ • PdM • ΧελϚʔαΫηε ◦ Ϧʔμʔީิ / ϝϯόʔ • ϢʔβʔαΫηε ◦ Ϧʔμʔީิ / ϝϯόʔ Findy Team+ • ৽نࣄۀ։ൃ • ηʔϧε • ΧελϚʔαΫηε • PdM ւ֎ਪਐ • ্ཱͪ͛ϝϯόʔ ◦ ϢʔβʔରԠ ◦ اըରԠ ࣾ௕ࣨ • ࣾ௕ࣨ௕ީิ • ࠾༻૊৫ਓࣄϦʔμʔ • ޿ใ୲౰ ίʔϙϨʔτଞ • CFOީิ • ܦӦاը • ๏຿ • ্৔४උ୲౰ • ࣄۀاը • ܦཧ ืूதͷϙδγϣϯ