Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ISUCON運営を支えるAmazon ECSとAurora Serverless v2 / AWS Dev Day 2022 Japan
Search
FUJIWARA Shunichiro
November 10, 2022
5
1.3k
ISUCON運営を支えるAmazon ECSとAurora Serverless v2 / AWS Dev Day 2022 Japan
https://aws.amazon.com/jp/events/devday/japan/
FUJIWARA Shunichiro
November 10, 2022
Tweet
Share
More Decks by FUJIWARA Shunichiro
See All by FUJIWARA Shunichiro
コードを書く隙間を見つけて生きていく技術/Findy 思考の現在地
fujiwara3
28
6.3k
fujiwara-ware OSSをひたすら紹介する/ya8-2024
fujiwara3
7
480
Amazon ECSで好きなだけ検証環境を起動できるOSSの設計・実装・運用 / YAPC::Hiroshima 2024
fujiwara3
22
7.1k
リアル事例から読み解くWebパフォーマンスチューニングの勘所/Offers web performance tuning
fujiwara3
4
1.5k
隙間家具OSS開発で『自分の庭』をつくる / kayac-andpad-event
fujiwara3
0
710
ISUCON作問入門/ ISUCON Summer Fes 2023
fujiwara3
2
1.6k
隙間家具職人が考えること/ecspresso meetup
fujiwara3
4
4.3k
MackerelとGrafana OnCallを連携してみた
fujiwara3
0
1.7k
Amazon ECS デプロイツール ecspresso 開発5年の歩み
fujiwara3
15
4.2k
Featured
See All Featured
Producing Creativity
orderedlist
PRO
338
39k
GraphQLとの向き合い方2022年版
quramy
33
12k
Building Your Own Lightsaber
phodgson
100
5.7k
Music & Morning Musume
bryan
41
5.6k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
242
1.2M
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
0
92
The Pragmatic Product Professional
lauravandoore
26
5.9k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
81
44k
Designing for Performance
lara
601
67k
Side Projects
sachag
451
41k
How to name files
jennybc
65
93k
The Power of CSS Pseudo Elements
geoffreycrofte
62
5k
Transcript
ISUCONӡӦΛࢧ͑Δ Amazon ECSͱ Aurora Serverless v2 ໘ന๏ਓΧϠοΫ ౻ݪढ़Ұ (@fujiwara) AWS
Dev Day 2022 Japan 2022.11.10
@fujiwara SREνʔϜ ISUCON 1,2,5,11 ! ༏উ4ճ ISUCON 3,8,12 ӡӦ(ग़)3ճ github.com/kayac/ecspresso
Amazon ECS σϓϩΠπʔϧ github.com/fujiwara/lambroll AWS Lambda σϓϩΠπʔϧ
ISUCONͱ LINEגࣜձࣾओ࠵ ʮͳΜͰ͋ΓʯͷWebαʔϏεύϑΥʔϚϯε νϡʔχϯάίϯςετ 2011։࢝ʙ2022Ͱ12ճ
ISUCONͱ WebΞϓϦέʔγϣϯ͕ಈ࡞͍ͯ͠ΔαʔόʔΠϯελϯε͕ ༻ҙ͞ΕΔ(1ʙ࠷େ5) Πϯελϯε֎෦͔ΒϕϯνϚʔΫ(HTTPϦΫΤετ)࣮ߦ ॲཧͰ͖ͨϦΫΤετͳͲ͔ΒείΞΛࢉग़ ڝٕ࣌ؒ8࣌ؒ ϕϯνϚʔΫείΞ͕Ұ൪ߴ͍νʔϜ͕༏উʂ
ISUCONʮͳΜͰ͋Γʯ ʮͳΜͰʯʹʮ༩͑ΒΕͨڝٕ༻Πϯελϯεͷ্Ͱ ಈ͘ͷͳΜͰ1มߋͯ͠Α͍ʯ • WebΞϓϦέʔγϣϯͷ࣮(ݴޠؚΊͯ) • ϛυϧΣΞOSͷઃఆ • ҙͷιϑτΣΞϛυϧΣΞͷಋೖ ֎ܗతͳڍಈΛҡ͍࣋ͯ͠ΕɺߴԽ͢Δख๏ʹ੍ݶͳ͍
1 Ұ෦ϨΪϡϨʔγϣϯʹ໌ه͞Εͨࣄ߲Λআ͘
ISUCON12֓ཁ ΦϯϥΠϯ༧બ 2022-07-23() 10:00-18:00 ࢀՃνʔϜ 698ʢֶੜ 98ʣ ࢀՃऀ߹ܭ 1657໊ʢֶੜ 246໊ʣ
ΦϯϥΠϯຊબ 2022-08-27() 10:00-18:00 ࢀՃνʔϜ 30ʢֶੜ 8ʣ ࢀՃऀ߹ܭ 84໊ʢֶੜ 22໊ʣ
ISUCONϙʔλϧαΠτ
ISUCONϙʔλϧͷࣄ ਃ͠ࠐΈ͔Β༧બ·Ͱ νʔϜɾϝϯόʔࢀՃड͚͚ ɹGitHub, DiscordΞΧϯτ࿈ܞ AWSಈ࡞ݕূ༻CloudFormation(CFn) ςϯϓϨʔτ ɹ࣮ࡍʹEC2ΛىಈɺsshՄೳ͔ݕূ Ϋʔϙϯ ɹAWS༷ɺ͘͞ΒΠϯλʔωοτ༷
ISUCONϙʔλϧͷࣄ ༧બ / ຊબ ڝٕ༻ΠϯελϯεͷIPΞυϨεΛѲ ɹϗετ্ͷagent͕ϙʔλϧͱ௨৴ ϕϯνϚʔΫ࣮ߦ ɹࢦఆͨ͠Πϯελϯεʹର࣮ͯ͠ߦ ɹϕϯνϚʔΫ݁ՌొAPI ڝٕதͷ݁ՌӾཡ
ɹϕϯνϚʔΫ݁ՌҰཡͱৄࡉ ɹશνʔϜͷείΞਪҠ
ISUCON12ϙʔλϧͷ࣮ https://github.com/isucon/portal ࣮OSS(MIT Lisense)ͳͷͰͲͳͨͰ ͑·͢2 @sorah ͕ISUCON10Ͱ࣮ վमɺҾ͖ܧ͍Ͱ3 2 υΩϡϝϯτ΄΅ͳ͍ͨΊ؆୯ʹಈ͔ͤΔΘ͚Ͱͳ͍…
ISUCON12ϙʔλϧͷߏ Rails + Shoryuken (SQS job worker) ɹSPAʹ͍ۙߏ ɹReact +
REST API (Protobuf) ɹDiscord, GitHub࿈ܞͳͲඇಉظjobͰ࣮ߦ ɹDiscord bot ϕϯνϚʔΧʔ ɹϙʔλϧAPI͔ΒjobΛऔಘ (Rustdaemon) ɹ→ ରΠϯελϯεʹϕϯνϚʔΫ࣮ߦ (GoCLI) ɹϕϯνϚʔΫ݁ՌΛϙʔλϧAPIʹૹ৴
ISUCON12ϙʔλϧAWS্ʹσϓϩΠ Amazon CloudFront Application Load Balancer (ALB) Amazon Simple Queue
Service (SQS) Amazon ElastiCache Redis Amazon ECS Fargate Aurora MySQL Serverless v2 [New!] ɹ2022-04 GA ɹͪΐ͏ͲͬͯΈ͔ͨͬͨʂ
ISUCON11→12Ͱͷ͓ͳมߋ • ϕϯνϚʔΧʔ͕ EC2 → ECS Fargate ʹ • ISUCON11
νʔϜ͝ͱʹ1ͷ EC2 Λ੩తׂΓͯ • ISUCON12 700νʔϜʹରͯ͠ Fargate 200 λεΫΛڞ༗ • RDS ͕ Aurora Serverless v2 ʹʂ
ISUCONϙʔλϧͷར༻ঢ়گ ࢀՃਃ͠ࠐΈ 220ʙ230νʔϜΛ3ճʹ͚ͯࢀՃొ 1. ॳճ 240ඵͰച 2. 2ճ 110ඵͰച 3.
3ճ 20ఔͰച ਃ͠ࠐΈ͕ຒ·Δ͚ͩ ۃʹΞΫηε͕ଟ͍
ISUCONϙʔλϧͷར༻ঢ়گ ༧બ ڝٕ։࢝࣌ࠁʹڝٕ༻ϖʔδ͕open ࢀՃऀ1700໊͕͍͍ͬͤʹΞΫηε CFnςϯϓϨʔτμϯϩʔυ ϕϯνϚʔΫ࣮ߦ ɹ༧બ࣌ 40,000ճ/8࣌ؒ = ฏۉ83ճ/
ɹϐʔΫ࣌120ճ/
ISUCONϙʔλϧʹAurora Serverless v2͕࠷ద? ΞΫηεͷ૿ݮ͕ۃΊͯݦஶ ΞΫηε͕ଟ͍ظؒಛʹ͍(ʙ࣌ؒ) 3ϲ݄ͷӡӦظؒதɺ΄ͱΜͲՋ ࠷খ Aurora Capacity Unit(ACU)
0.5Ͱे ΞΫηε͕૿ݮ͢ΔλΠϛϯάಡΊΔɺ͕… աڈͷܦݧ͕ͳ͍ͨΊ૿Ճ͕ಡΈͮΒ͍ ݟੵΓΛޡΔͱΓͳ͍ = োͷՄೳੑ͕͋Δ
ISUCONϙʔλϧʹAurora Serverless v2͕࠷ద! ෛՙ͕૿ݮ͢ΔλΠϛϯά = ਃ͠ࠐΈڝٕ։࢝ɺऴྃ࣌ → ӡӦ͕ଟ ࣄલεέʔϧΞοϓɺࣄޙεέʔϧμϯͷखΛֻ͚ͨ͘ͳ͍ ίετൺֱత͍҆
ΠϯελϯελΠϓ USD/h USD/month db.t3.small 0.063 45.99 Serverless v2 (0.5 ACU) 0.10 73.00 db.t4g.medium 0.113 82.49
࣮ࡍͷར༻ঢ়گΛϝτϦΫεͰಡΈղ͘ υΩϡϝϯτ Aurora Serverless v2ͷॏཁͳAmazon CloudWatchϝτϦΫε3 ͔Β ServerlessDatabaseCapacity ʹ ʼ
ΠϯελϯεϨϕϧͷϝτϦΫεͱͯ͠ɺݱࡏͷDBΠϯελϯεͷ༰ྔͰ ද͞ΕΔ ACU Λใࠂ͠·͢ 3 https://docs.aws.amazon.com/ja_jp/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.setting-capacity.html#aurora- serverless-v2.parameters-based-on-scaling
ࢀՃొୈ2ظ 70ඵͰ230νʔϜ͕ച 42k/min = 700req/secఔ ɹSPAͳͷͰ΄΅ͯ͢API Fargate 16 task (1
vCPU 4GiB) ɹCPU ΄΅100%ʹ ɹFargateࣄલʹtaskखಈઃఆ άϥϑCloudFrontͷϦΫΤετ
ࢀՃొୈ2ظ Queries & ServerlessDatabaseCapacity Queries 30 qps → 600 qps
ServerlessDatabaseCapacity 0.5 → 2 Կ͠ͳ͍Ͱෛՙʹैͯ͠ͳ͠ 1ͰͷϐʔΫͬͱӶ͍4 4 ϝτϦΫεऔಘ࣌ʹ5ཻͷ͔͍ͬͯ͠ͳ͔ͬͨͨΊ
༧બ։࢝࣌ Queries & ServerlessDatabaseCapacity 9:40 ͡Θ͡ΘͱΞΫηε͕૿͑Δ 10:00 1700໊͕Ұ੪ʹΞΫηε 10:00ͷΫΤϦ ɹ9:55ͱൺֱͯ͠10ഒ
ɹ9:59ͱൺֱͯ͠5ഒ ServerlessDatabaseCapacity 0.5 → 3ʹॠ࣌ʹै ઌճΓ্͕͍ͯͬͯ͠ΔΑ͏ʹʁ
༧બ։࢝࣌ SelectLatency & ServerlessDatabaseCapacity ։࢝લ͍Ζ͍ΖͳΫΤϦ͕গ → ϨΠςϯγ҆ఆ͠ͳ͍ ։࢝ޙҰ੪ʹಉछͷΞΫηε → ϨΠςϯγ͕҆ఆ
→ Queries, ACU͕ٸ૿ͯ҆͠ఆ
༧બ1Λ௨ͯ͠ 10:00 ɹQueries 1.2k/sec ɹServerlessDatabaseCapacity 2.5 18:00 ɹQueries 2.8k/sec ɹServerlessDatabaseCapacity
8.5 20࣌Ҏ߱ʹͶ͍ͯΔͷӡӦʹΑΔ ࢼͷӨڹ(ޙड़) FargateΕ͍ͯͨ (ΦʔτεέʔϧͰ͍͍ײ͡ʹ૿͑ͨ)
༧બ1Λ௨ͯ͠ SelectLatency & ServerlessDatabaseCapacity 10:00ʙ17:00·ͰLatency҆ఆ 17:00ʙ18:00ϙʔλϧͷڍಈ (ൃߦ͢ΔΫΤϦ)͕มΘͬͨӨڹͷมԽ
༧બऴྃޙͷӡӦࢼ ڝٕ༻ΠϯελϯεΛશ࠶ىಈޙ ڝٕதͷείΞ͕࠶ݱͰ͖Δ͔ࢼ 700νʔϜͷϕϯνϚʔΫҰؾʹೖ × 3ճ CLIΛ࣮ߦ͍ͯͨ͠ମײͰ 1ճ࠷ॳඵҾ͔͔ͬΔ → ͦͷޙεϜʔζʹྲྀΕͨ
༧બऴྃޙͷӡӦࢼ ࠷ॳͷϨΠςϯγ͕େ͖Ί CapacityएׯΕͯ૿Ճ 2, 3ճCapacity͕མ͖͍ͪͬͯͳ͍ →εϜʔζʹै͍ͯ͠Δ "Aurora Serverless v2Ͱͷ ύϑΥʔϚϯεͱεέʔϦϯά"5
ʼ ݱࡏͷ༰ྔ͕େ͖͍΄ͲɺεέʔϧΞοϓ͕ ͘ͳΓ·͢ 5 https://docs.aws.amazon.com/ja_jp/AmazonRDS/latest/AuroraUserGuide/ aurora-serverless-v2.setting-capacity.html
εέʔϧΠϯ࣌ͷڍಈ Queries(੨) & ServerlessDatabaseCapacity(ᒵ) 18:00ʙ ঃʑʹԼ͕Δ͕2.5ఔΛҡ࣋ 21:00ʙ 0.5·Ͱݮগ ૿Ճ࣌ෛՙʹԠͯ͡Ұؾʹ্͕Δ ݮগ࣌ෛՙ͕ݮ͙ͬͯ͢ʹ࠷·Ͱ
Լ͕Βͣ ͋Δఔҡ࣋ͨ͋͠ͱঃʑʹԼ͢Δ
·ͱΊ Aurora Serverless v2ΛISUCON12ϙʔλϧʹ࣮ઓೖ͠·ͨ͠ ಉҰੑೳͰprovisionedͳΠϯελϯεͱൺֱ͢ΔͱߴՁ͕ͩ ࣗಈͰෛՙैͯ͘͠ΕΔͨΊӡ༻ίετ͕ۃΊ͍ͯ ීஈՋɺʹಥൃෛՙ͕ൃੜ͢ΔΠϕϯτ༻ͷαΠτӡӦ ʹۃΊ࣮ͯ༻తͳಛੑ