Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
マルチクラウドで構築する大規模解析サービス/戦略とポイントについて/
Search
tik-son
February 15, 2018
Technology
28k
9
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
マルチクラウドで構築する大規模解析サービス/戦略とポイントについて/
デブサミ2018でお話したこと。
http://event.shoeisha.jp/devsumi/20180215/session/1621/
tik-son
February 15, 2018
Transcript
55 slides
© 2018 PLAID,Inc. ϚϧνΫϥυͰߏங͢ΔେنղੳαʔϏε ઓུͱϙΠϯτʹ͍ͭͯ 'FCSVBSZ 1-"*% *OD /BPIJLP5BLFNVSB
© 2018 PLAID,Inc. 1SPpMF 2011ɹಉࢤࣾେֶֶ෦ଔۀ 2011͔Β2014·ͰɺNECʹͯΫϥυαʔϏεͷ্ཱͪ͛ʹैࣄ σʔληϯλʔͷϑΝγϦςΟ͔ΒωοτϫʔΫ/αʔόͷཧϨΠϠʔ͔ΒԾ ϨΠϠʔ·Ͱͷઃܭ/ߏங/ӡ༻Λ୲ 2014ɹϓϨΠυʹೖࣾ େنղੳαʔϏεKARTEͷཪଆΛΠϯϑϥ͔ΒόοΫΤϯυ·Ͱ෯͘୲
ଜɹঘ
© 2018 PLAID,Inc. 8IBU`T1-"*%
© 2018 PLAID,Inc. 4IPFT" 4IPFT# d d 4)011*/($"35
๚தͷϢʔβʔΛ ϦΞϧλΠϜղੳ ݸʑͷϢʔβʔʹ߹Θͤͨ ίϛϡχέʔγϣϯΛఏڙ 8IBU`T,"35&
© 2018 PLAID,Inc. 8IBU`T,"35& ๚தͷϢʔβʔΛϦΞϧλΠϜʹղੳ
© 2018 PLAID,Inc. 8IBU`T,"35& ϦΞϧλΠϜʹղੳͨ݁͠ՌΛͱʹ༷ʑͳΞΫγϣϯΛग़͚͠Δ͜ͱ͕Ͱ͖Δ
© 2018 PLAID,Inc. ಋೖΫϥΠΞϯτ ղੳUU DBͷ༰ྔ τϥοΩϯάΠϕϯτ TB events /
sec 6000 1430+ 500 ղੳ࣌ؒ ඵ 0.x UU 12.5ԯ ྲྀ௨ֹۚ ԁ / 5000ԯ 4UBUTPG,"35& ΫϥΠΞϯτ
© 2018 PLAID,Inc. Event Cost $PTUTPG,"35& 2લ ? ӡ༻ʹܞΘΔ Engineer
ݱࡏ
© 2018 PLAID,Inc. Event Cost *OGSBTUSBUFHZPG,"35& Ͱ͖Δ͚ͩγϯάϧΫϥυͰ γϯϓϧͳߏΛΉ ϓϩόΠμʹͱΒΘΕͣ ϚϧνΫϥυʹߏΛΉ
ઓུͷมߋ 2લ ӡ༻ʹܞΘΔ Engineer ݱࡏ
© 2018 PLAID,Inc. ϚϧνΫϥυΛ͍͜ͳͨ͢Ίͷ4ͭͷϙΠϯτ
© 2018 PLAID,Inc. શମߏ
© 2018 PLAID,Inc. 1. ֤ϓϩόΠμʔͷ͍͍ͱ͜औΓΛ͢Δ
© 2018 PLAID,Inc. ༷ʑͳϓϩόΠμ/αʔϏε͕ଘࡏ - ૯߹తͳϓϩόΠμɺҰಛԽܕͷϓϩόΠμ - ϓϩόΠμͷ࣭ͷ্ ϓϩόΠμ/αʔϏεʹಘҙ͕͋Δ -
AWS ͱ GCP ಘҙΛ͍͍ͱ͜ͲΓ͢ΔͱγϯάϧϓϩόΠμͰͷγεςϜΑΓڧ͘ͳͬͨ
© 2018 PLAID,Inc. Analyze Ωϡʔ ϦΞϧλΠϜղੳͷྲྀΕ ߦಈϩάऩूϓϩηε ෛՙίϯτϩʔϧͷͨΊͷ ΩϡʔΠϯά ղੳϓϩηε
ղੳσʔλอଘ ղੳσʔλͷऔಘ DB Track
© 2018 PLAID,Inc. Analyze Ωϡʔ ϦΞϧλΠϜղੳͷྲྀΕ ߦಈϩάऩूϓϩηε ෛՙίϯτϩʔϧͷͨΊͷ ΩϡʔΠϯά ղੳϓϩηε
ղੳσʔλอଘ ղੳσʔλͷऔಘ Track DB
© 2018 PLAID,Inc. Analyze Ωϡʔ ϦΞϧλΠϜղੳͷྲྀΕ ߦಈϩάऩूϓϩηε ෛՙίϯτϩʔϧͷͨΊͷ ΩϡʔΠϯά ղੳϓϩηε
ղੳσʔλอଘ - ϨΠςϯγʔ͚ͩͲϑϧϚωʔδυͰͳ͍ - ಥવͷϝϯςφϯε - ύϑΥʔϚϯεͷݶք ղੳσʔλͷऔಘ Track DB
© 2018 PLAID,Inc. - DBͷ༷͕KARTEͱ૬ੑ͕ѱ͔ͬͨ - ΧλϩάεϖοΫ௨ΓʹύϑΥʔϚϯεग़ͳ͍ - ͏΄ͲΤΫεϙωϯγϟϧʹίετ͕͔͔Δ ϦΞϧλΠϜղੳͷྲྀΕ
ߦಈϩάऩूϓϩηε ෛՙίϯτϩʔϧͷͨΊͷ ΩϡʔΠϯά ղੳϓϩηε ղੳσʔλอଘ Ωϡʔ Analyze ղੳσʔλͷऔಘ DB Track - ϨΠςϯγʔ͚ͩͲϑϧϚωʔδυͰͳ͍ - ಥવͷϝϯςφϯε - ύϑΥʔϚϯεͷݶք
© 2018 PLAID,Inc. Analyze Ωϡʔ ϦΞϧλΠϜղੳͷྲྀΕ ߦಈϩάऩूϓϩηε ෛՙίϯτϩʔϧͷͨΊͷ ΩϡʔΠϯά ղੳϓϩηε
ղੳσʔλอଘ ղੳσʔλͷऔಘ Track DB
© 2018 PLAID,Inc. redis enterpriseΛΫϥυͰఏڙ͍ͯ͠ΔαʔϏε ಛ - ߴ͍εέʔϥϏϦςΟ - ্༷σʔλ༰ྔɺίϯϐϡʔλϦιʔεͷ੍ݶ͕ͳ͘ɺෛՙʹԠͯࣗ͡ಈͰεέʔϧ
- ϨΠςϯγɺߴύϑΥʔϚϯε - Latency 0.1ms - ReadOps 3kOps/sec, WriteOps 11kOps/sec - ϑϧϚωʔδυαʔϏε - Δ͜ͱॳظઃఆͷΈ - ߴՄ༻ੑ - ೋؒେ͖ͳো͕ͳ͍ - DBͷσʔλྔʹΑΔैྔ՝ۚɻωοτϫʔΫίετͰ՝ۚ͞Εͳ͍ɻ ϨΠςϯγɺεέʔϥϒϧͳΩϡʔΛίετͰར༻Ͱ͖ΔΑ͏ʹ SFEJTDMPVE
© 2018 PLAID,Inc. Analyze ϦΞϧλΠϜղੳͷྲྀΕ ղੳσʔλͷऔಘ ߦಈϩάऩूϓϩηε ෛՙίϯτϩʔϧͷͨΊͷ ΩϡʔΠϯά ղੳϓϩηε
ղੳσʔλอଘ Track
© 2018 PLAID,Inc. GoogleͷαʔϏεͰར༻͍ͯ͠Δٕज़ΛΫϥυͰఏڙ ಛ - ߴ͍εέʔϥϏϦςΟ - ্༷σʔλ༰ྔͷ੍ݶ͕ͳ͍ɻ140TB -
ύϑΥʔϚϯεΞοϓͷͨΊͷεέʔϧΞοϓखಈ͕ͩɺμϯλΠϜͳ͘ॠ࣌ʹ࣮ߦ - ߴՄ༻ੑ - ೋؒେ͖ͳো͕ͳ͍ - ϨΠςϯγɺߴύϑΥʔϚϯε - Latency -20ms - Readεϧʔϓοτ 1.3GB/sec, Writeεϧʔϓοτ 2.3MB/sec - ϑϧϚωʔδυαʔϏε $MPVE#JHUBCMF ΧλϩάεϖοΫҎ্ͷߴ͍ύϑΥʔϚϯε
© 2018 PLAID,Inc. Analyze - ύϑΥʔϚϯε/҆ఆੑ্͕ - αʔϏεར༻ίετ͕30%ݮ - ӡ༻ʹ͔͚Δ࣌ؒͷݮ
ߦಈϩάऩूϓϩηε ෛՙίϯτϩʔϧͷͨΊͷ ΩϡʔΠϯά ղੳϓϩηε ղੳσʔλอଘ ղੳσʔλͷऔಘ Track
© 2018 PLAID,Inc. 2. ϓϩόΠμͷোΛผϓϩόΠμͰΧόʔ͢Δ
© 2018 PLAID,Inc. ͲΕ͚ͩૉΒ͍͠αʔϏεͰোى͖Δ - 2017/3ͷS3ͷେنো(https://aws.amazon.com/jp/message/41926/) - 3࣌ؒɺUS-EAST-1ͷS3͕͑ͳ͘ͳΓɺଟ͘ͷαʔϏε͕μϯ - S399.99ͷSLA
োͷதʹγϯάϧΫϥυͰରԠ͠Α͏͕ͳ͍ͷଘࡏ͢Δ - ωοτϫʔΫͳͲڞ௨෦ͷো - Ϧʔδϣϯ/κʔϯͷ֓೦Λ࣋ͨͳ͍ϚωʔδυαʔϏεͷো
© 2018 PLAID,Inc. KARTE͓٬༷ͷαΠτͷҰ෦ʹͳΔͷͰՄ༻ੑͱͯॏཁ - ྫ͑chat ϚϧνΫϥυʹΑͬͯ͜ͷΑ͏ͳোΛ্ख͑͘Δ͜ͱ͕Ͱ͖͍ͯΔ
© 2018 PLAID,Inc. https://status.cloud.google.com/incident/compute/16020#5177473553661952 2࣌ؒɺόοΫΤϯυͷεςʔλεʹؔͳ͘ ϦΫΤετͷ1/3Ͱ502ͷϨεϙϯεΛฦͯ͠͠·ͬͨͱ͍͏LBͷো
© 2018 PLAID,Inc.
© 2018 PLAID,Inc. ߦಈϩάऩूपΓͷߏ ීஈRoute53ͷTrafficFlowͰ GCP : AWS = 8
: 2 Ͱ໊લղܾ
© 2018 PLAID,Inc. ߦಈϩάऩूपΓͷߏ ো࣌Route53ͷTrafficFlowͰGCP : AWS = 0 :
10 Ͱ໊લղܾ
© 2018 PLAID,Inc. ֤ϓϩόΠμʔࣅͨΑ͏ͳαʔϏεΛఏڙ͍ͯ͠ΔͷͰɺ༷ʑͳॴͰ࣮ݱՄೳ શͯͰ͜ͷߏΛͱΔͷ͍͠ͷͰɺಋೖ෦ʹ͍ͭͯݟۃΊ͕ඞཁ - γεςϜͱͯ͠ͷॏཁ͞ - ࣮ݱͷқ ཁ݅ʹԠͯ͡ɺͲ͜·Ͱ࡞ΓࠐΉ͔ߟ͑Δ
- ଟগίεύ͕ѱͯ͘ಈ͚͍͍ - खಈͰΓସ͑Ε͍͍ ϦΞϧλΠϜղੳʹؔΘΔॏཁͳ෦ ग़དྷΔ͚ͩෳͷϓϩόΠμʔͰಈ͔ͤΔΑ͏ʹ͍ͯ͠Δ GCS S3 CloudCDN CloudFront
© 2018 PLAID,Inc. 3. ϓϩόΠμ·ͨ͗ͷ௨৴ʹҙ͢Δ
© 2018 PLAID,Inc. 1. ௨৴ίετ͕ඦഒʹͳͬͨ 2. ωοτϫʔΫͷεϧʔϓοτ͕ஶ͘͠Լͨ͠
© 2018 PLAID,Inc. ௨৴ίετ͕ඦഒʹͳͬͨ - ϦΞϧλΠϜղੳͰͷBigtableͱEC2ͷ௨৴Ͱର - BigtableͷසൟͳRead/WriteʹΑΓίετ૿ NBJO
© 2018 PLAID,Inc. ௨৴ίετ͕ඦഒʹͳͬͨ ผϓϩόΠμʔͷαʔϏεͷ௨৴ͷίετʹҙ - ϦΞϧλΠϜղੳͰͷBigtableͱEC2ͷ௨৴Ͱର - BigtableͷසൟͳRead/WriteʹΑΓίετ૿ -
Bigtableಉ͡zone͔Βͷ௨৴ίετ͔͔Βͳ͍ - Bigtableͱಉ͡zoneʹGCEΛߏங - ෭࣍తͳޮՌͱͯ͠ɺ҆ఆੑ/Latency্͕ - AWSGCPͷόοΫΞοϓܥʹ NBJO TUBOECZ NBJO
© 2018 PLAID,Inc. ωοτϫʔΫͷεϧʔϓοτ͕ஶ͘͠Լͨ͠ - GCPͱAWSଓͷͨΊͷVPN෦ - GCE͔ΒMongo(EC2)ͷେྔͷRead - VPNͷଳҬ͕ανΓϘτϧωοΫ
- 1.5Gps/1VPNτϯωϧ(GCP) - ଳҬʹಛʹ੍ݶͳ͠(AWS)
© 2018 PLAID,Inc. ผϓϩόΠμͷ71/ܦ༝ͷ௨৴ͷଳҬʹҙ ωοτϫʔΫͷεϧʔϓοτ͕ஶ͘͠Լͨ͠ - GCPʹReadReplicaΛ࡞ΓɺVPNͷ௨৴ྔݮ - ෳVPNτϯωϧʹIPϧʔςΟϯάͰෛՙࢄ -
ଳҬΛࢹ͠ɺඞཁʹԠͯ͡τϯωϧΛՃ - GCPͱAWSଓͷͨΊͷVPN෦ - GCE͔ΒMongo(EC2)ͷେྔͷRead - VPNͷଳҬ͕ανΓϘτϧωοΫ - 1.5Gps/1VPNτϯωϧ(GCP) - ଳҬʹಛʹ੍ݶͳ͠(AWS) IUUQUFDIQMBJEDPKQBXT@HDQ@NVMUJDMPVE@
© 2018 PLAID,Inc. ผϓϩόΠμʔͷαʔϏεͷ௨৴ͷίετ ผϓϩόΠμʔͷVPNܦ༝ͷ௨৴ͷଳҬ NBJO TUBOECZ
© 2018 PLAID,Inc. 4. ෳͷϓϩόΠμʔΛ౷ҰతͳΠϯλʔϑΣʔεͰӡ༻
© 2018 PLAID,Inc. ΫϥυαʔϏεͱ͍ͬͯࣗͨͪͰ໘Λݟͳ͍ͱ͍͚ͳ͍෦͕͋Δ - ߏཧ - ࢹ - ΞϓϦέʔγϣϯͷσϓϩΠ
ͳͲ ӡ༻Λิॿ͢ΔαʔϏε֤ϓϩόΠμͰͦΕͧΕಠࣗͷϑΥʔϚοτ - ྫ͑ߏཧ DeploymentManager CloudFormation
© 2018 PLAID,Inc. DeploymentManager
© 2018 PLAID,Inc. DeploymentManager CloudFormation
© 2018 PLAID,Inc. ౷ҰతͳΠϯλʔϑΣʔε
© 2018 PLAID,Inc. ࣗͨͪͰ։ൃʁ - ϓϩόΠμ͕૿͑ΔͭͲେ͖ͳ։ൃ͕ඞཁ - ରߏཧɺΞϓϦέʔγϣϯͷσϓϩΠɺࢹͳͲ - ίετେͰϓϩόΠμΛؾܰʹ૿ͤͳ͍
ಉ͡՝ײΛ๊͍͑ͯΔਓ݁ߏ͍Δ OSSɺαʔυύʔςΟͷαʔϏεΛ͏·͘ར༻͢Δ ౷ҰతͳΠϯλʔϑΣʔεΛͲ͏࡞Δ͔ʁ
© 2018 PLAID,Inc. ߏཧ ΞϓϦέʔγϣϯͷσϓϩΠ ࢹ
© 2018 PLAID,Inc. ϓϩϏδϣϯ WerckerΛτϦΨʔ ߏఆٛϑΝΠϧΛ PR & Merge ߏཧͷΈ
© 2018 PLAID,Inc. σϓϩΠఆٛϑΝΠϧ ΛPR & Merge WerckerΛτϦΨʔ Roer ΞϓϦέʔγϣϯͷσϓϩΠͷΈ
σϓϩΠఆٛΛϓϩϏδϣϯ
© 2018 PLAID,Inc. ఆٛࡁΈͷσϓϩΠͷapiΛୟ͘ σϓϩΠͷ࣮ߦ deployίϚϯυ Roer ΞϓϦέʔγϣϯͷσϓϩΠͷΈ σϓϩΠఆٛϑΝΠϧ ΛPR
& Merge WerckerΛτϦΨʔ σϓϩΠఆٛΛϓϩϏδϣϯ
© 2018 PLAID,Inc. ALERT MetricsͷӾཡ ి ௨ ALERT/Dashboardͷఆٛ Metricsͷू ࢹͷΈ
© 2018 PLAID,Inc. ߏཧ ΞϓϦέʔγϣϯͷσϓϩΠ ࢹ
© 2018 PLAID,Inc. ·ͱΊ
© 2018 PLAID,Inc. αʔϏεͷͱγϯάϧΫϥυͷݶք͔Β ʮϓϩόΠμʹͱΒΘΕͣϚϧνΫϥυʹߏΛΉʯઓུʹ ϙΠϯτ4ͭ 1.ϓϩόΠμʔͷ͍͍ͱ͜औΓΛ͢Δ 2.ϓϩόΠμͷোΛผϓϩόΠμͰΧόʔ͢Δ 3.ϓϩόΠμ·ͨ͗ͷ௨৴ʹҙ͢Δ 4.ෳͷϓϩόΠμʔΛ౷ҰతͳΠϯλʔϑΣʔεͰӡ༻͢Δ
© 2018 PLAID,Inc. ࠷ޙʹ
© 2018 PLAID,Inc. IUUQTDPEF[JOFKQBSUJDMFDPSOFS
© 2018 PLAID,Inc. IUUQUFDIQMBJEDPKQBXT@HDQ@NVMUJDMPVE@
© 2018 PLAID,Inc. IUUQTXXXXBOUFEMZDPNDPNQBOJFTQMBJE
More Decks by tik-son
See All by tik-son
Microsoft の AI 技術 をベースに構成される Craft Copilot
tikson
0
630
KARTE CraftにおけるLLM活用~Craft Functions Copilot~
tikson
1
2.3k
AWS とKARTE で作る User State Driven System / user-state-driven-system-with-aws-and-karte
tikson
0
2.6k
User State Driven System with Amazon EventBridge & KARTE
tikson
1
4k
PLAID’s Journey with multi cloud
tikson
1
4.1k
KARTEを支えるAutoScaling~SpotInstance編
tikson
0
1.9k
Other Decks in Technology
See All in Technology
信頼性を犠牲にせず、PR マージからデプロイまで人の介入を完全にゼロにするまでの道のり
knwoop
2
720
【AWS reInvent 2026 関西組 事前勉強会】二度あることは三度ある~調子に乗りそうな3回目の私を戒めたいです~
ttelltte
0
110
サーバーラックは作る時代! 3Dプリンターで作る10インチラックのススメ
jacopen
2
470
専門家に使ってもらうAIプロダクトをつくる - 病院の医事課向けDPCコーディング支援システムの設計と評価
hagino3000
4
870
Agent Routerで作るマルチテナント 推論基盤 #CNDW2026 / Building Multi-Tenant Inference Platform with Agent Router #CNDW2026
everpeace
3
390
個人ASを取得して、自宅Kubernetes上のVMをグローバルIPで公開した話
munenick
0
210
Re:ゼロから始めるシェアハウスのペットな鯖
silmin_
0
250
AIネイティブ組織の作り方。1人のPOとAIから始める、自社での実験
ama_ch
2
1.2k
cdkdがどんだけ早いか、実際に動かして見てみよう
smt7174
1
130
コピーオンライト型ファイルシステムと ランダム書き込み
sat
PRO
2
130
2026-11-11 Tamachi.sre#7 PaCにインフラリソースの命を救われました。 / PaC saves the infra resource life
masasuzu
1
210
IoTのデータ保存をSREの観点で考える
grimoh
1
110
Featured
See All Featured
GraphQLとの向き合い方2022年版
quramy
50
15k
A designer walks into a library…
pauljervisheath
211
25k
Mozcon NYC 2025: Stop Losing SEO Traffic
samtorres
1
600
WENDY [Excerpt]
tessaabrams
15
41k
Odyssey Design
rkendrick25
PRO
3
960
How to Grow Your eCommerce with AI & Automation
katarinadahlin
PRO
2
350
Bioeconomy Workshop: Dr. Julius Ecuru, Opportunities for a Bioeconomy in West Africa
akademiya2063
PRO
1
460
Context Engineering - Making Every Token Count
addyosmani
9
1.3k
Navigating Weather and Climate Data
rabernat
0
590
Max Prin - Stacking Signals: How International SEO Comes Together (And Falls Apart)
techseoconnect
PRO
0
530
Optimizing for Happiness
mojombo
378
72k
Measuring Dark Social's Impact On Conversion and Attribution
stephenakadiri
2
330