Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Serverless??そんなことよりDBだ!! / We should know Datab...
Search
Masashi Terui
November 03, 2018
Technology
3
3.8k
Serverless??そんなことよりDBだ!! / We should know Databases before Serverless
JAWS Festa 2018の資料です
https://jft2018.jaws-ug.jp/session/729/
Masashi Terui
November 03, 2018
Tweet
Share
More Decks by Masashi Terui
See All by Masashi Terui
What is Jeffy ?? - Why did we create a new framework
marcyterui
2
890
Serverlessの今とこれから / Everything will be Serverless
marcyterui
7
3.5k
見せてやろう…!! Serverlessの本当の力を…!! / Let’s show you the real power of serverless…!!
marcyterui
6
2.6k
re:Invent 2018でServerlessの世界は何が変わったか / What has changed in the world of Serverless at re:Invent 2018??
marcyterui
4
3k
実践サーバレス + α / Serverless Practices
marcyterui
2
620
Serverlessを極めるためにDynamoDBデータモデリングを極めよう / Let’s become the master of DynamoDB Data Modeling to become the master of Serverless
marcyterui
12
17k
マイクロサービスを成功させるためのサーバーレスアーキテクチャ設計とNoSQLデータモデリング / Serverless Architecting and NoSQL Data Modeling for Successful Microservices
marcyterui
13
6.9k
何故私達はサーバレスアプリケーションをイベントドリブンで作るべきなのか / Why should we make serverless application as event driven?
marcyterui
8
1.3k
The theory of data modeling to become the friend of DynamoDB
marcyterui
2
230
Other Decks in Technology
See All in Technology
AIのAIによるAIのための出力評価と改善
chocoyama
0
510
AIの最新技術&テーマをつまんで紹介&フリートークするシリーズ #1 量子機械学習の入門
tkhresk
0
130
Welcome to the LLM Club
koic
0
140
知識を整理して未来を作る 〜SKDとAI協業への助走〜
yosh1995
0
150
Wasm元年
askua
0
100
Agentic Workflowという選択肢を考える
tkikuchi1002
1
390
Clineを含めたAIエージェントを 大規模組織に導入し、投資対効果を考える / Introducing AI agents into your organization
i35_267
4
1.4k
Microsoft Build 2025 技術/製品動向 for Microsoft Startup Tech Community
torumakabe
1
210
Create a Rails8 responsive app with Gemini and RubyLLM
palladius
0
140
Кто отправит outbox? Валентин Удальцов, автор канала Пых
lamodatech
0
280
[TechNight #90-1] 本当に使える?ZDMの新機能を実践検証してみた
oracle4engineer
PRO
3
140
エンジニア向け技術スタック情報
kauche
0
110
Featured
See All Featured
For a Future-Friendly Web
brad_frost
179
9.8k
How STYLIGHT went responsive
nonsquared
100
5.6k
Side Projects
sachag
455
42k
Large-scale JavaScript Application Architecture
addyosmani
512
110k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
48
2.8k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
4
200
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
35
2.3k
Navigating Team Friction
lara
187
15k
Designing for humans not robots
tammielis
253
25k
The Power of CSS Pseudo Elements
geoffreycrofte
77
5.8k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
181
53k
Transcript
SERVERLESSʁʁ ͦΜͳ͜ͱΑΓDBͩʂʂ MASASHI TERUI @ JAWS FESTA OSAKA 2018
SERVERWORKS CO.,LTD. + FREELANCER • Serverless Oji-san • Serverless Framework
Plugin Developer • Serverlessconf Tokyo 2016,2017,2018 speaker • Remote worker (in Sapporo-shi, Hokkaido) • http://marcy.hatenablog.com/entry/2018/08/04/231241 MASASHI TERUI ARCHITECT / DEVELOPER
SERVERLESSͬͯ·͔͢ʁ DynamoDBͬͯ·͔͢ʁ
• ͖ • ී௨ • ݏ͍ • LambdaͱRDSͷ૬ੑ͕ѱ͍͔Βํͳ͖͘߹͍ͬͯΔ DynamoDB͖Ͱ͔͢ʁ
ํ͕ͳ͍͔ΒDynamoDB…… ຊRDBΛ͍͍͚ͨͲ
RDBʹ͑ͯΔΜ͡Όͳ͍Ͱ͔͢ʁʁ NoSQL͕͑ͳ͍Μ͡Όͳͯ͘
RDBͷ͜ͱͪΌΜͱ͔ͬͯ·͔͢ʁʁ ͦͦ
• ίωΫγϣϯϞσϧ • Ӭଓత • ίωΫγϣϯϓʔϦϯά • ηΩϡϦςΟ • ΈࠐΈͷऑ͍ೝূ
• VPCͷΞΫηείετʢENIੜॲཧʣ LambdaͱRDSͷ૬ੑ͕Կނѱ͍ͷ͔આ໌Ͱ͖·͔͢ʁ ͱͯOLTPͰ͑ͳ͍(ඵʙेඵ)
DynamoDBͷ͜ͱ͔ͬͯ·͔͢ʁʁ ͡Ό͋
σʔλϕʔεͷ͜ͱ͔ͬͯ·͔͢ʁʁ ͱ͍͏͔
• ςʔϒϧઃܭͱ͔ΠϯσοΫεઃܭͬͯͲ͏Δͷʁ • ACIDτϥϯβΫγϣϯແ͍ͷʹେৎͳͷʁ • ݁Ռ߹ੑͬͯେৎͳͷʁ • ্ख͘ੑೳΛҾ͖ग़͢ʹͲ͏͢Εྑ͍ʁ DynamoDBͷΑ͋͘Δෆ҆
• ςʔϒϧઃܭͱ͔ΠϯσοΫεઃܭͬͯͲ͏Δͷʁ • ACIDτϥϯβΫγϣϯແ͍ͷʹେৎͳͷʁ • ݁Ռ߹ੑͬͯେৎͳͷʁ • ্ख͘ੑೳΛҾ͖ग़͢ʹͲ͏͢Εྑ͍ʁ DynamoDBͷΑ͋͘Δෆ҆ ͔Βͳ͍͔Βෆ҆ʹͳΔ
• AWS͕ఏڙ͢ΔNoSQLαʔϏε • KeyʹΑΔશҰகͱIndex୳ࡧ͕Մೳ • ϑϧϚωʔδυ • ແ੍ݶͷࣗಈ֦ுετϨʔδʢ͚ͬͨͩʣ • ΠϯελϯεͷཧෆཁɺWrite/Readͷੑೳ͚ͩΛࢦఆ͢Δ
• ͰɺΩϟύγςΟͷࣄલ༧͕ඞཁͳαʔϏεServerless͡Όͳ͍ͬͯCNCF͞Μ͕ݴͬɾɾɾ͋ɺ୭͔དྷͨΈ͍ͨ DynamoDBͷجຊ
DynamoDBͲΜͳ ”AWSαʔϏε” ͔ Ͱͳ͘
DynamoDBͲΜͳ ”σʔλϕʔε” ͔ ͬͯ·͔͢ʁ
• Consistent Hashing • Vector clocks with reconciliation during reads
• Sloppy Quorum and hinted handoff • Anti-entropy using Merkle trees • Gossip-based membership protocol and failure detection Amazon’s Dynamo ৄ͘͠Γ͍ͨͳΒʢBetterͰ͋ͬͯMustͰͳ͍ʣ https://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
• ΩʔͷϋογϡͰࢄ → Partition Key • ͦͷࢄͨ͠தͰIndex͕ߏங͞Ε͍ͯΔ → Sort Key
• όοΫάϥϯυͰߋ৽ΛޙউͪͰಉظ • ݁Ռ߹ੑϕʔε • ॻ͖ࠐΈɾಡΈࠐΈͷQuorumͰ߹ੑ͕ܾ·Δ • ΩʔΛࢦఆͨ͠ಡΈࠐΈڧ߹ੑ͕બՄೳ ͑Δ͖ϙΠϯτ
Partition KeyͷϋογϡͰࢄ ͋ɺ͜ΕRDB(Hash Index)Ͱʢ͈́
• Partition Keyʹ࿈൪Λ͏ͷԿͷҙຯͳ͍͜ͱ͕͔Δ • ࿈൪ΧϯλʔΛ࡞ΔͱPartition͕ภΔ • Max + 1ͳΜͯҎͯͷ֎ •
ཧతʹҰҙͱͳΔଐੑ͕͋ΔͳΒͦΕΛ͏ • ͳ͚ΕUUIDͷΑ͏ͳিಥՄೳੑ͕ۃখͳϥϯμϜͰྑ͍ • SortͰ͖ͳ͍ͷͰSnowflakeͷΑ͏ͳॱংੑཁΒͳ͍ • User IDCognitoͰೝূ͍ͯ͠ΔͳΒCognito͕ൃߦ͢ΔIDΛ͏ • ΞΫηε੍ޚʹ͑Δ ͜ͷલఏΛ౿·͑Δͱ
• 3000RCU, 1000WCU, Index Size 10GB (2018-10-31ݱࡏ) • ͲΕ͔ҰͭͰա͢ΔͱPartitionׂ͕͞ΕΔ •
ׂ͞ΕΔͱ֬อͨ͠Ϣχοτ͕ۉʹׂΓৼΒΕΔ • εϩοτϦϯά࣌ͷҰ࣌తͳόʔετ͕͋Δ • ݶఆతͳόʔετ • ༨Λճ͢ (Adaptive Capacity) Partition͝ͱͷΩϟύγςΟ
Partition KeyͰࢄɺSort Keyʁ
B+TREE INDEX B-treeͳͷ͔B+treeͳͷ͔ެද͞Ε͍ͯͳ͍͚Ͳ Range Keyͱ͔ݺͿ͘Β͍͔ͩΒ…Ͷʁ ͋ɺ͜ΕRDBͰʢ͈́
TARGETING QUERY B+treeͷߏΛ͍ͬͯΕ Έ߹ΘͤΩʔͷΠϯσοΫε͕ແͯ͘ ͜͏Εྑ͍͜ͱ͕͔Δ
Partition KeyͰࢄͨ͠தͰSort KeyͰྻ ͜ͷΠϝʔδΛ࣋ͭ͜ͱ͕ͱͯॏཁ
• Partition KeyͰࢄͨ͠ΠϯελϯεʹIndex͕ߏங͞ΕΔ • ҰͭͷΠϯελϯεͰಉظతʹIndexߋ৽͞ΕΔ • ڧ߹ੑ͕બՄೳ • Primary Sort
KeyͷϦʔϑϊʔυʹ࣮σʔλͷΞυϨε͕ ֨ೲ͞Ε͍ͯΔΠϝʔδʢͱࢥΘΕΔʣ • LSI࣮σʔλΛ࣋ͨͳ͍ͷͰॏෳՄೳ Primary Sort Key, LSI
GSI࣮ςʔϒϧͷࣹӨ
• ू߹ʹ͓͚ΔࣹӨ • GSIͷPartition KeyΛݩʹࢄ ʢߋ৽ඇಉظͰద༻ʣ • ϦʔϑϊʔυʹଐੑͱPrimary Key ʢৄ͘͠ޙड़ʣ
• ιʔτࡁΈͷσʔλͷ෦ෳ ʢҟೝΊΔʣ ࣹӨ ࡶͳMySQLύϑΥʔϚϯενϡʔχϯά from yoku0825 https://www.slideshare.net/yoku0825/mysql-57449062/62
• KEYS_ONLY: Primary KeyͷͷΈ • INCLUDE: Primary Keyͷ + બͨ͠ଐੑ
• ALL: શͯͷଐੑ • GSIʹؚΊͨଐੑPrimary KeyΛҾ͖͢ඞཁ͕ͳ͘ͳΔ • ಡΈࠐΈͷޮԽ • ߋ৽ίετͱͷτϨʔυΦϑ • MySQLPostgreSQLͰݴ͏Covering Index / Index Only ScanͷΠϝʔδ GSI͕ࣹӨ͢ΔଐੑͷΦϓγϣϯ
ࣹӨͱ͍͏͜ͱ 1:Nͷ͕ؔ͜ͷΑ͏ʹղܾͰ͖Δ
• ϝΠϯςʔϒϧͱผͷPartitionͰࢄ͞ΕΔ • ߋ৽ඇಉظͰө • ݁Ռ߹ੑͬͯେৎͳͷʁ • RDSͩͬͯRead ReplicaಡΜͩΒ݁Ռ߹ੑͰ͢Αʁʁ •
GSIͷΫΤϦ͍͍ͩͨRead Replicaʹ͛ΔΑ͏ͳΫΤϦ GSIʹର͢ΔΫΤϦ݁Ռ߹ੑ
͋ɺ͜ΕRDBͰʢԿճʁ
ΈΛΕ࣮ʹγϯϓϧ શ෦RDBͰطʹ͋ͬͨΈͰ͢Αʁʁ
☓ DynamoDBบ͕͋Δ ◦ RDBͷบ͕͋Γ͗͢Δ →γϯϓϧ͗͢ΔDynamoDBͱͷΪϟοϓ͕ڧͯͦ͘͏ײ͡Δ͚ͩ
RDBʹચ͞ΕͯΔΜ͡Όͳ͍Ͱ͔͢ʁʁ ͱ͍͏͔ͦͷRDBͷ͜ͱ͢ΒͪΌΜͱ͔ͬͯ͋͛ͯ·͔ͨ͠ʁ
RDBͷ͜ͱ͔ͬͯ·͔ͨ͠ʁ ͱ͍͏͔ɺͦͷେ͖ͳ
DynamoDBͷσʔλϞσϦϯά ͰͦΕΛ౿·͑ͯ
ͦΕσʔλͷஔΛܾΊΔ͜ͱ ҟೝΊΔʂʂ
σʔλϞσϦϯάͷߟ͑ํͷϙΠϯτ ஔΛܾΊΔͨΊͷ
• εΩʔϚϨεͷҙຯΛߟ͑Δ • List, Map͕࣋ͯΔ͜ͱʁ • ޙ͔ΒଐੑΛγʔϜϨεʹՃͰ͖Δ͜ͱʁ • A. ͳΜͰಥͬࠐΊΔ͜ͱ
• Partition KeyͰࢄ • ςʔϒϧΛ͚ͳͯ͘ෛՙࢄ͢Δ • ۃ͢Δͱ͚Δҙຯ͕ͳ͍ɺ͚ͳ͍ํ͕ΩϟύγςΟ͕ཧ͍͢͠ • “ઃܭ͕༏ΕͨΞϓϦέʔγϣϯͰɺඞཁͳςʔϒϧ 1 ͭͷΈͰ͢” • GSIͷݸ੍ݶ(5ݸ)ͳͲ͋ΔͷͰۀυϝΠϯຖʹҰͭ͘Β͍ͷؾ࣋ͪ ςʔϒϧͷ͚ํ
• RDB • ·ͣεΩʔϚΛߟ͑Δ • ਖ਼نԽ • ͦΕʹରͯ͠Ͳ͏ΞΫηε͢Δ͔(SQL)ߟ͑Δ • DynamoDB
• εΩʔϚϨε • ඇਖ਼نԽ • ·ͣΞΫηεΛߟ͑ͯɺͦΕʹ߹ΘͤͨσʔλΛ࡞Δ • ΞϓϦέʔγϣϯͱσʔλϞσϧಉ࣌ʹઃܭ͢Δ RDBͱΞϓϩʔν͕શ͘ҟͳΔ
ACIDτϥϯβΫγϣϯ͕ແ͍ͷͲ͏͢Δͷʁ ͱ͍͑
• ݪࢠੑ (Atomicity) • Ұ؏(߹)ੑ (Consistency) • ಠཱੑ (Isolation) •
Ӭଓੑ (Durability) ͦͦACIDͬͯͳΜ͚ͩͬʁ ͜ΕΒΛ୲อ͢Δ͜ͱ͕తͰ͋ͬͯ τϥϯβΫγϣϯγεςϜͰ͋Δඞཁແ͍
• RDB • ਖ਼نԽ͍ͯ͘͠ͱςʔϒϧͷجຊతʹ૿͑Δ • ཧతʹ͔Εͨςʔϒϧ(Ϩίʔυ)ؒͷ߹ੑΛऔΔΈ͕͋Δ • ϦϨʔγϣϯγοϓ • ֎෦Ωʔ੍
• ACIDτϥϯβΫγϣϯ • DynamoDB • ඇਖ਼نԽͯ͠1ͭͷ࣮ମʹ͍ͭͯ1ͭͷΞΠςϜʹऩΊΔ • 1ΞΠςϜͷߋ৽ΞτϛοΫ • ͖݅ॻ͖ࠐΈΛָͬͨ؍తϩοΫ ACIDͷ୲อ
• PutItem • ΞΠςϜΛ·Δ͝ͱߋ৽͢Δɺແ͔ͬͨΒ࡞Δ • Get -> ॻ͖͑ -> Putͩͱ߹ੑ่͕ΕΔʢՄೳੑ͕͋Δʣ
• UpdateItem • ΞΠςϜΛ෦ߋ৽ɺແ͔ͬͨΒ࡞Δ • List, Mapͷ෦ՃɺআͳͲՄೳ • ଐੑͷΛར༻ͨ͠ߋ৽Մೳʢ͋Δଐੑʹ+1͢Δ) • ଐੑͷΛར༻͖ͨ݅͠ॻ͖ࠐΈ DynamoDBͷߋ৽ૢ࡞
ඇਖ਼نԽ͢ΔͱಡΈࠐΈ͕ͭΒ͍ ϑϧεΩϟϯ͢Δ͔͠ͳ͍ શ෦GSIషΔͷʁ
CQRSͰ͢Α ͦ͜Ͱ
• ॻ͖ࠐΉσʔλͱಡΈࠐΉσʔλಉ͡Ͱ͋Δඞཁແ͍ • ݁Ռ߹ੑΛड͚ೖΕͯඇಉظͰಡΈࠐΈσʔλΛ࡞ΕΔ • CommandͷྃΛͬͯQuery༻σʔλΛ࡞ΕΔ • DynamoDB Streams͕Ϛον͢Δ •
CQRSΛૉʹ࣮͢Δͱ·ͣKinesis Streams͚ͩͲ • Materialized View CQRS (Command Query Responsibility Segregation)
ARCHITECTURE PATTERN AppSync͕͜Ε͔ΒॏཁͳҐஔΛΊΔͣ
DATA ATOMICITY MicroserviesతͳΠϝʔδͰ͜ͷݪ୯Ґ͕ॏཁ σʔλετΞɾςʔϒϧͷ͚ํͰͳ͍ ލ͕͍ͤͨͳΒMaterialized ViewΛ࡞Δ User Profile User Status
Name Email Read/Write Search Read/Write
͜ΜͳΠϝʔδ 1ͭͷGSIͰ༷ʑͳݕࡧ͕Ͱ͖Δ
• ෳͷ࣮ମʹލΔτϥϯβΫγϣϯ • DynamoDB୯ମͩͱ͜͜Ͳ͏ͯͭ͠Β͍ • τϥϯβΫγϣϯϨϕϧͷ࠷ߴSelializable • ͭ·ΓྻԽ • Kinesis
StreamsͰྻԽ • ίέΔ͔͠Εͳ͍ͷͰႈʹ࣮͢Δ • ݅ॻ͖ࠐΈͰطʹߦͬͨॲཧΛSkip͢Δͷႈ ͱ͍͑ݶք͋Δ
• Kinesis StreamsͰಉ࣌ଓΛίϯτϩʔϧ͢Ε ผʹRDSͬͯԿͷͳ͍Θ͚Ͱ • ಉ࣌ଓ → ShardͱಉͰݻఆ • LambdaجຊWarm
Start • ͭ·ΓɺENIͷԆͱ΄΅ແԑ • σϓϩΠShardม͑ͨλΠϛϯάͰى͖͏Δ͚ͲͦͦඇಉظͳͷͰʢ͈́ ͱ͍͏͔
ADVANCED TIPS ͕࣌ؒ͋Ε͍͔ͭ͘͝հ
• DynamoDBͷIndexSparse • Indexଐੑ͕ଘࡏ͠ͳ͍߹ɺIndexʹͦͷσʔλؚ·ͳ͍ • ಛఆͷΞΠςϜͷΈʹଘࡏ͢ΔଐੑΛࢦఆ͢Δ͜ͱͰ Index༰ྔ͕େ͖͘ѹॖͰ͖Δ Sparse Index
SortཁΒͳ͍ GSIʹ҉ͰPrimary Keyؚ͕·ΕΔ
• Α͋͘Δ֊ߏΛ࣋ͭσʔλ • ৫ߏ • ωετͨ͠ΧςΰϥΠζ • ͳͲͳͲ • ݕࡧΛߟ͑ͳ͚Ε֊ߏΛ·Δͬͱ1ΞΠςϜʹԡ͠ࠐΉ
• Sort KeyͰ֊Λදݱ͢Δ Hierarchical data
Sort KeyͰલํҰகݕࡧ ͋ɺ͜ΕSQLΞϯνύλʔϯͰʢ͈́ ઃܭ͕ࡶͳͷ͝Ѫᇷ
• Itemʹ”Version”ͷΑ͏ͳଐੑΛ࣋ͨͤΔ • ଐੑͷΛར༻͖ͨ݅͠ॻ͖ࠐΈ • ConditionExpression=“Version = 1” • UpdateExpression=“ADD
Version 1” • Version = 1Ͱͳ͚ΕΤϥʔ͕ฦΔ • Version = 1ͳΒ2ʹߋ৽͞ΕΔ όʔδϣχϯάͱָ؍తϩοΫ
• తʹݴͬͯόΠϒϧ • ͜ΕΛಡΜͰશͯཧղͰ͖ͨΒҰਓલʂʂ • https://docs.aws.amazon.com/ja_jp/amazondynamodb/latest/ developerguide/best-practices.html DynamoDB Best Practices
“DynamoDB ދͷר” [ݕࡧ] http://marcy.hatenablog.com/entry/2018/07/31/213705 ҰԠະͳͷͰ·ͩॻ͔͘
“LET’S BECOME THE FRIEND OF RDB” THANKS!! “LET’S BECOME THE
FRIEND OF DYNAMODB” “LET’S BECOME THE FRIEND OF SERVERLESS”