Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
月間10億PVから学んだMongoDBアンチパターン
Search
yujiosaka
November 11, 2014
Technology
19
5.5k
月間10億PVから学んだMongoDBアンチパターン
ZenClerkが月間10億PVを支えるまでの 過程で経験したアンチパターンを紹介
yujiosaka
November 11, 2014
Tweet
Share
More Decks by yujiosaka
See All by yujiosaka
I was understanding WASM all wrong! 🤯
yujiosaka
2
250
Machine Learning with JavaScript
yujiosaka
0
160
JavaScriptでも機械学習がやりたかった話
yujiosaka
2
430
ヘッドレスChromeでクローラを作った後の話
yujiosaka
3
650
俺が最初にヘッドレスChromeでクローラ作った 事になんねーかな
yujiosaka
4
1.2k
『XXX』のための管理画面
yujiosaka
1
1.3k
Enjoy Deep Learning by JavaScript
yujiosaka
1
330
ひたすら楽してディープラーニング
yujiosaka
20
13k
technology x business
yujiosaka
3
550
Other Decks in Technology
See All in Technology
DETR手法の変遷と最新動向(CVPR2025)
tenten0727
2
1.4k
React ABC Questions
hirotomoyamada
0
490
Amazon CloudWatch を使って NW 監視を行うには
o11yfes2023
0
170
AIエージェント開発手法と業務導入のプラクティス
ykosaka
5
1.5k
Devinで模索する AIファースト開発〜ゼロベースから始めるDevOpsの進化〜
potix2
PRO
8
3.5k
AWS Control Towerを 数年運用してきての気づきとこれから/aws-controltower-ops-tips
tadayukinakamura
0
160
クラウド開発環境Cloud Workstationsの紹介
yunosukey
0
180
AI AgentOps LT大会(2025/04/16) Algomatic伊藤発表資料
kosukeito
0
140
地味にいろいろあった! 2025春のAmazon Bedrockアップデートおさらい
minorun365
PRO
1
280
Making a MIDI controller device with PicoRuby/R2P2 (RubyKaigi 2025 LT)
risgk
1
290
YOLOv10~v12
tenten0727
4
960
技術者はかっこいいものだ!!~キルラキルから学んだエンジニアの生き方~
masakiokuda
2
270
Featured
See All Featured
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
45
7.2k
Done Done
chrislema
183
16k
Bash Introduction
62gerente
611
210k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
30
2k
Become a Pro
speakerdeck
PRO
27
5.3k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
jQuery: Nuts, Bolts and Bling
dougneiner
63
7.7k
Statistics for Hackers
jakevdp
798
220k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
47
2.7k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
Scaling GitHub
holman
459
140k
Why Our Code Smells
bkeepers
PRO
336
57k
Transcript
݄ؒ10ԯPV͔ΒֶΜͩ MongoDBΞϯνύλʔϯ db tech showcase ౦ژ 2014 speaker: “Yuji Isobe”
history: [ “violinist” “engineer” “security specialist” “socket.io contributor” “startup member
of ZenClerk ” ] name: “Yuji Isobe”
questions: -> MongoDBΛ༻ͨ͜͠ͱ͕͋Γ·͔͢ʁ -> ຊ൪ڥͰMongoDBΛ༻͍ͯ͠·͔͢ʁ
goal: -> ZenClerk͕݄ؒ10ԯPVΛࢧ͑Δ·Ͱͷ աఔͰܦݧͨ͠ΞϯνύλʔϯΛհ -> ओʹύϑΥʔϚϯενϡʔχϯάͷ ->ʮ݄ؒ10ԯPVΛࢧ͑ΔMongoDBʯͷͦͷઌ IUUQXXXTMJEFTIBSFOFUZVKJPTBLBQWNPOHPEC
None
ϦΞϧλΠϜ
ϦΞϧλΠϜੳ Ϗοάσʔλ
db.ZenClerk .stats() -> ݄ؒ10ԯPV -> ඵؒ400PV -> ಉ࣌ଓ5ສ -> ݄ؒ10TBอଘ
architecture: front-end: [ “AngularJS” “Ruby on Rails” “MySQL” “memcached” “AWS”
] back-end: [ “Node.js” “MongoDB” “socket.io” “Redis” “AWS” ]
architecture: front-end: [ “AngularJS” “Ruby on Rails” “MySQL” “memcached” “AWS”
] back-end: [ “Node.js” “MongoDB” “socket.io” “Redis” “AWS” ]
i2.2xlarge m1.large m3.large db.ZenClerk .aggregate() r3.large
ʙ1000ສPV mogngod
1000ສʙ1ԯPV delayed replica replica set
1ԯPVʙ10ԯPV replica set shard mongos replica set shard replica set
shard delayed replica delayed replica delayed replica mongoc
͜͜·ͰΞϓϦέʔγϣϯͷมߋ ΄ͱΜͲͳ͠ʢҰ෦ΫΤϦΛ࠷దԽʣ
we.use(“MongoDB”).explain() -> εΩʔϚϨε -> τϥϯβΫγϣϯෆཁ ʢ࠷ݶͷΞτϛοΫૢ࡞ʣ -> ॊೈͳΫΤϦ -> ༰қͳεέʔϧΞτ
-> Node.jsͱͷ૬ੑͷྑ͞
we.use(“MongoDB”).explain(true) -> ಋೖͷ༰қ͞ʢϗεςΟϯάαʔϏε༗ແʣ -> ֶशͷ༰қ͞ʢΤϯδχΞҎ֎ʹͱͬͯʣ
ࠓ͢ͷɾɾɾ ϕετϓϥΫςΟε Ξϯνύλʔϯ
ͪ͜Βʂ ϕετϓϥΫςΟε Ξϯνύλʔϯ
Ͱͦͷલʹɾɾɾ
ϕετϓϥΫςΟε
εΩʔϚσβΠϯ -> TwitterͷλΠϜϥΠϯΛอଘ͢Δ -> RDBMSͳΒਖ਼نԽ -> MongoDBͷϕετϓϥΫςΟεʁ
-> ಡΈࠐΈॏࢹ -> 1ΫΤϦ1ώοτ -> ίϝϯτʹԠͯ͡ ॻ͖ࠐΈ͕ॏ͘ͳΔ υΩϡϝϯτΛຒΊࠐΉ session: _id:
“session_id” device: “iPhone” page_views: [ {url: “/items/1”} ] timeline: _id: “yujiosaka” tag: “#dbtechshowcase” tweets: [ {text: “MongoDB Rocks!”} ]
-> ॻ͖ࠐΈॏࢹ -> 2ΫΤϦN+1ώοτ -> ॻ͖ࠐΈεέʔϧ͢Δ υΩϡϝϯτΛຒΊࠐ·ͳ͍ session: _id: “session_id”
device: “iPhone” page_views: [ {url: “/items/1”} ] timeline: _id: “yujiosaka” tag: “#dbtechshowcase” ! tweet: _id: 1234 user_id: “yujiosaka” text: “MongoDB Rocks!”
timeline: _id: “yujiosaka-20141111” tag: “#dbtechshowcase” tweets: [ {text: “MongoDB Rocks!”}
] -> όϥϯεܕ -> 1ΫΤϦώοτ -> ॻ͖ࠐΈ1·Ͱ ͔͠ॏ͘ͳΒͳ͍ ϋΠϒϦου
εΩʔϚϨε ≠ εΩʔϚఆ͕ٛෆཁ RDBMSΑΓબࢶ͕ଟ͍͘͠ tips: IUUQXXXNPOHPECDPNQSFTFOUBUJPOTTDIFNBEFTJHOTDBMF
Ξϯνύλʔϯ
ΠϯσοΫεฤ ΫΤϦฤ Ξοϓσʔτฤ ΞʔΩςΫνϟฤ
ᶃΠϯσοΫεͷషΓա͗
MongoDBϝϞϦͷ͍ํ͕ ͋·Γݡ͋͘Γ·ͤΜ
before: user = new mongoose.Schema first: {type: String, required: true,
index: true} last: {type: String, required: true, index: true} city: {type: String, required: true, index: true} ... created_at: {type: Date, default: Date.now, index: true} ! ॻ͖ࠐΈ͕٘ਜ਼ʹͳͬͯಡΈࠐΈ͕ૣ͚Ε͍͍͡ΌΜʂ
after: user = new mongoose.Schema first: {type: String, required: true}
last: {type: String, required: true} city: {type: String, required: true} ... created_at: {type: Date, default: Date.now} ! ॻ͖ࠐΈ͕٘ਜ਼ʹͳͬͯಡΈࠐΈ͕ૣ͚Ε͍͍͡ΌΜʂ
ಡΈࠐΈɺΠϯσοΫε͕ ϝϞϦʹΒͳ͚Ε͘ͳΔ
tips: όʔδϣϯ2.8ʹPluggable Storage Engine͕༧ఆ͞Ε͍ͯΔͷͰɺ ݡ͍Τϯδϯ͕બΔΑ͏ʹͳΔ https://blog.compose.io/the-coming-of-the-mongodb-storage-engines/
ᶄจࣈྻͷΠϯσοΫε
ΠϯσοΫεͷߏ -> RDBMSͱ΄ͱΜͲಉ͡ -> ΠϯσοΫεʹB-TreeΛ࠾༻ -> B-TreeৗʹฏߧΛอ͍ͬͯΔ
จࣈྻͷΠϯσοΫε -> ϗοτσʔλ͕ࢄ -> ϝϞϦޮ͕ѱ͍
࣌ؒɾObjectIdͷΠϯσοΫε -> ϗοτσʔλ͕ूத -> ϝϞϦޮ͕ྑ͍
tips: Ͱ͖Δ͚ͩObjectIdΛ͍·͠ΐ͏ ޙ͔Βͷஔ͖͑େมͰ͢
ΠϯσοΫεฤ ΫΤϦฤ Ξοϓσʔτฤ ΞʔΩςΫνϟฤ
ᶅϑΟʔϧυΛࢦఆ͠ͳ͍ΫΤϦ
Ϣʔβʔใͷ͏ͪ ࢢใ͚͕ͩཉ͍͠ user: {_id: “yujiosaka”, first: “Yuji”, last: “Isobe”, city:
“Tokyo”} index: {_id:1, city: 1} // compound index
ϑΟʔϧυࢦఆͳ͠ -> SQLͰͷʮselect *ʯ -> ෆཁͳωοτϫʔΫίετ -> ෆཁͳσΟεΫΞΫηε var user
= db.users.findOne({_id: “yujiosaka”});
ϑΟʔϧυࢦఆ͋Γ -> SQLͰͷʮselect cityʯ -> ωοτϫʔΫίετͷઅ -> covered index ͕׆༻Ͱ͖Δ
var user = db.users.findOne({_id: “yujiosaka”}, {city: 1});
tips: covered indexΛ༗ޮ׆༻͢Εɺ σΟεΫͷΞΫηεҰͳ͠ʹ ϝϞϦ͚͔ͩΒใΛಡΈࠐΊΔ
ᶆίʔϧυσʔλͷΫΤϦ
MongoDBͷಛ -> ϗοτσʔλͷΞΫηεߴ -> ίʔϧυσʔλͷΞΫηε͍ -> શσʔλͷΞΫηεۤख
ZenClerkʹ͓͚Δूܭ -> 5͝ͱʹσʔλΛཁ -> 1࣌ؒ͝ͱʹ౷ܭσʔλΛूܭ -> 1͝ͱʹ౷ܭσʔλΛूܭ
tips: ৗʹϗοτσʔλΛҙࣝͨ͠ ͍ํΛ͠·͠ΐ͏
ᶇϓϥΠϚϦʹภͬͨΫΤϦ
ϨϓϦΧηοτ -> ΫΤϦΛࢄͤ͞Δ͜ͱ͕Ͱ͖Δ -> ηΧϯμϦʹΫΤϦΛ͛Δ͔ ϓϩάϥϚ͕ܾΊΒΕΔ -> σϑΥϧτͰϓϥΠϚϦʹ͛ΒΕΔ
σϑΥϧτ -> ϓϥΠϚϦ͔Βಡ·ΕΔ -> ࠷৽ͷσʔλ͕อো͞ΕΔ var user = db.users.find({city: “Tokyo”});
ηΧϯμϦΛࢦఆ -> ηΧϯμϦ͕͋ΕηΧϯμϦ͔Βɺ ͳ͚ΕϓϥΠϚϦ͔Βಡ·ΕΔ -> ࠷৽ͷσʔλอো͞Εͳ͍ var user = db.users.find({city:
“Tokyo”}) .readPref("secondaryPreferred")
tips: σʔλ͕࠷৽Ͱͳͯ͘Α͍߹ ಡΈࠐΈΛηΧϯμϦʹ͚Δ͜ͱͰ ΫΤϦ͕ࢄ͢Δ IUUQTTQFBLFSEFDLDPNNPOHPECQSBDUJDBM TDBMJOHBOETIBSEJOHFMJPUIPSPXJU[HFO
ΠϯσοΫεฤ ΫΤϦฤ Ξοϓσʔτฤ ΞʔΩςΫνϟฤ
ᶈඇޮͳΞοϓσʔτ
Α͘ݟ͔͚Δίʔυᶃ -> ඞͣΫΤϦ͕࣮ߦ͞ΕΔ -> ແବͳωοτϫʔΫίετ -> ඇΞτϛοΫͳॲཧ var user =
db.users.findOne({_id: “yujiosaka”}); user.count += 1; user.save();
Α͘ݟ͔͚Δίʔυᶃ -> updateͱ$incΛͬͯ·ͱΊΒΕΔ -> 1ճͷΞοϓσʔτͰࡁΉ -> ΞτϛοΫͳॲཧ db.users.update({_id: “yujiosaka”}, {$inc:
{count: 1}});
Α͘ݟ͔͚Δίʔυᶄ -> σʔλ͕ݟ͔ͭͬͨΒߋ৽͠ɺݟ͔ͭΒͳ͔ͬͨΒ࡞͢Δ -> Ұݟෳࡶͳίʔυ var user = db.users.findOne({_id: “yujiosaka”});
if (user) { user.count += 1; user.save(); } else { db.users.insert({_id: “yujiosaka”, count: 1}); }
-> {upsert: true} Λ͏ͱ·ͱΊΒΕΔ -> ΑΓγϯϓϧͳίʔυ Α͘ݟ͔͚Δίʔυᶄ db.users.update({_id: “yujiosaka”}, {$inc:
{count: 1}}, {upsert: true});
ᶉංେԽ͢ΔυΩϡϝϯτ
Ϛεͷ࠲ඪσʔλΛ อଘ͢Δ behavior: points: [{x: 151, y: 100} {x: 151,
y: 179} {x: 151, y: 266} … {x: 151, y: 340}]
MongoDBͷۤख -> ϑΟʔϧυ͕ංେԽ͢ΔΞοϓσʔτ͕͍ -> σʔλ͕ҰఆͷαΠζʹऩ·Βͳ͘ͳΔͱ σΟεΫͰσʔλͷҠಈ͕ى͖Δ -> RedisͰMongoDBͷۤखΛิ͏
Redis -> ΦϯϝϞϦܕͷKVS -> ಡΈࠐΈɾॻ͖ࠐΈ͕ૣ͍ -> جຊతʹσʔλͷӬଓԽ͠ͳ͍ -> σʔλܕ͕ඇৗʹ๛ ʢϦετɾηοτɾϋογϡʣ
RedisΛΘͳ͍߹ db.col.update({_id: 1234}, {$push: {points: {x: 151, y: 100}}); db.col.update({_id:
1234}, {$push: {points: {x: 151, y: 179}}); db.col.update({_id: 1234}, {$push: {points: {x: 151, y: 266}}); … db.col.update({_id: 1234}, {$push: {points: {x: 151, y: 340}}); -> ࠲ඪ͕Ҡಈ͢ΔͨͼʹMongoDBʹॻ͖ࠐ·ΕΔ -> αΠζ͕ऩ·ΓΒͳ͍ͱɺσʔλͷҠಈ͕ى͖Δ
RedisΛͬͨ߹ db.col.insert({points: [ {x: 151, y: 100}, {x: 151, y:
179}, {x: 151, y: 266}, … {x: 151, y: 340} ]}); -> Ұ͔͠MongoDBʹॻ͖ࠐ·ͳ͍ -> υΩϡϝϯυͷαΠζ͜ΕҎ্େ͖͘ͳΒͳ͍
tips: ͯ͢ΛMongoDB͚ͩͰΖ͏ͱͤͣɺ ۤखͳ෦ଞͷٕज़Ͱิ͏͜ͱ͕େ
ΠϯσοΫεฤ ΫΤϦฤ Ξοϓσʔτฤ ΞʔΩςΫνϟฤ
ᶊ٧ΊࠐΈա͗DB
σʔλΛҰͭͷDBʹ٧ΊࠐΜͰ ྑ͍͜ͱʢࠓͷͱ͜Ζʣ͋·Γͳ͍
ͳͥDBΛ͚Δͷ͔ʁ -> Reader-WriterϩοΫ͕ΘΕ͍ͯΔ -> 2.2.0Ҏ߱DB୯ҐͰϩοΫ͢Δ -> ϚγϯΛ͚ΕϝϞϦ͕૿͑Δ
ZenClerk ʹ͓͚ΔDBͷ༻్ -> ϝΠϯσʔλʢHotʣ -> ϝΠϯσʔλʢColdʣ -> ੜσʔλ -> ܭࢉ݁Ռσʔλ
-> ౷ܭσʔλ -> αΠτଐੑσʔλ
tips: όʔδϣϯ2.8ʹdocument-level locking 3.0Ҏ߱ʹpartitioned joins͕༧ఆ͞Ε͍ͯΔ ʢޙऀ͋·Γ͋ͯʹ͠ͳ͍ํ͕ແͦ͏ʣ http://www.slideshare.net/tetsutarowatanabe/mongodb-world-2014-37081258
ᶃΠϯσοΫεͷషΓա͗ ᶄจࣈྻͷΠϯσοΫε ᶅϑΟʔϧυΛࢦఆ͠ͳ͍ΫΤϦ ᶆίʔϧυσʔλͷΫΤϦ ᶇϓϥΠϚϦʹภͬͨΫΤϦ ᶈඇޮͳΞοϓσʔτ ᶉංେԽ͢ΔυΩϡϝϯτ ᶊ٧ΊࠐΈա͗DB Ξϯνύλʔϯ͓͞Β͍
Ξϯνύλʔϯʹ ؕΒͳ͍ͨΊʹ
̏ͭͷࢹ -> ϦΞϧλΠϜࢹ -> ݟฦͨ͢Ίͷࢹ -> ͍ΫΤϦͷࢹ
ϦΞϧλΠϜࢹ
-> ComposeʢMongoHQʣμογϡϘʔυ -> ݱࡏͷ݈߁νΣοΫ༻ -> ࣾʹઐ༻σΟεϓϨΠΛઃஔ -> ҟৗ͕͋Εʮ୭͔Mongoࡴͨ͠ʁʯ ϦΞϧλΠϜࢹ
ݟฦͨ͢Ίͷࢹ
-> MMSμογϡϘʔυ -> ఆظతͳ݈߁அ -> োൃੜ࣌ʹݟฦͯ͠ ɹʮͳΜͰ͜ΜͳϩοΫߴ͘ͳͬͯΜͷʁʯ ݟฦͨ͢Ίͷࢹ
͍ΫΤϦͷࢹ
-> explain(true) -> system.profile.find() -> Dex -> Compose (MongoHQ) Slow
Queries ͍ΫΤϦͷࢹ
tips: ObjectIdΛͬͨΫΤϦʹ 100msҎ্͔͔ͬͨΒةݥ৴߸
my.presentation.aggregate() -> MongoDBબࢶ͕ଟ͍ -> Ξϯνύλʔϯ͍͔ͭ͘ଘࡏ͢Δ -> ΞϯνύλʔϯʹؕΒͳ͍ͨΊʹࢹ͢Δ -> ࠷৽ใͷΩϟονΞοϓॏཁ
͍͞͝ʹ
we.are “hiring! :)” https://www.zenclerk.com/recruit.html
any.questions?