Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Serverless??そんなことよりDBだ!! / We should know Datab...
Search
Masashi Terui
November 03, 2018
Technology
3
3.7k
Serverless??そんなことよりDBだ!! / We should know Databases before Serverless
JAWS Festa 2018の資料です
https://jft2018.jaws-ug.jp/session/729/
Masashi Terui
November 03, 2018
Tweet
Share
More Decks by Masashi Terui
See All by Masashi Terui
What is Jeffy ?? - Why did we create a new framework
marcyterui
2
860
Serverlessの今とこれから / Everything will be Serverless
marcyterui
7
3.4k
見せてやろう…!! Serverlessの本当の力を…!! / Let’s show you the real power of serverless…!!
marcyterui
6
2.5k
re:Invent 2018でServerlessの世界は何が変わったか / What has changed in the world of Serverless at re:Invent 2018??
marcyterui
4
2.9k
実践サーバレス + α / Serverless Practices
marcyterui
2
610
Serverlessを極めるためにDynamoDBデータモデリングを極めよう / Let’s become the master of DynamoDB Data Modeling to become the master of Serverless
marcyterui
12
17k
マイクロサービスを成功させるためのサーバーレスアーキテクチャ設計とNoSQLデータモデリング / Serverless Architecting and NoSQL Data Modeling for Successful Microservices
marcyterui
13
6.8k
何故私達はサーバレスアプリケーションをイベントドリブンで作るべきなのか / Why should we make serverless application as event driven?
marcyterui
8
1.3k
The theory of data modeling to become the friend of DynamoDB
marcyterui
2
220
Other Decks in Technology
See All in Technology
スケールアップ企業のQA組織のバリューを最大限に引き出すための取り組み
tarappo
4
960
ルートユーザーの活用と管理を徹底的に深掘る
yuobayashi
6
730
ISUCONにPHPで挑み続けてできるようになっ(てき)たこと / phperkaigi2025
blue_goheimochi
0
140
移行できそうでやりきれなかった 10年超えのシステムを葬るための戦略 / phper-kaigi-2025-ryu
carta_engineering
0
690
KCD Brazil '25: Enabling Developers with Dapr & Backstage
salaboy
1
120
OCI見積もり入門セミナー
oracle4engineer
PRO
0
120
バックエンドエンジニアによるフロントエンドテスト拡充の具体的手法
kinosuke01
1
730
PostgreSQL Unconference #52 pg_tde
nori_shinoda
1
220
AIエージェントキャッチアップと論文リサーチ
os1ma
6
1.2k
Redefine_Possible
upsider_tech
0
270
どっちの API SHOW?SharePoint 開発における SharePoint REST API Microsoft Graph API の違い / Which API show? Differences between Microsoft Graph API and SharePoint REST API
karamem0
0
110
17年のQA経験が導いたスクラムマスターへの道 / 17 Years in QA to Scrum Master
toma_sm
0
410
Featured
See All Featured
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
356
30k
A designer walks into a library…
pauljervisheath
205
24k
What’s in a name? Adding method to the madness
productmarketing
PRO
22
3.4k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
44
7.1k
Designing for Performance
lara
606
69k
BBQ
matthewcrist
88
9.5k
Git: the NoSQL Database
bkeepers
PRO
429
65k
Six Lessons from altMBA
skipperchong
27
3.7k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
30
2.3k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
46
2.4k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
8
700
Speed Design
sergeychernyshev
28
860
Transcript
SERVERLESSʁʁ ͦΜͳ͜ͱΑΓDBͩʂʂ MASASHI TERUI @ JAWS FESTA OSAKA 2018
SERVERWORKS CO.,LTD. + FREELANCER • Serverless Oji-san • Serverless Framework
Plugin Developer • Serverlessconf Tokyo 2016,2017,2018 speaker • Remote worker (in Sapporo-shi, Hokkaido) • http://marcy.hatenablog.com/entry/2018/08/04/231241 MASASHI TERUI ARCHITECT / DEVELOPER
SERVERLESSͬͯ·͔͢ʁ DynamoDBͬͯ·͔͢ʁ
• ͖ • ී௨ • ݏ͍ • LambdaͱRDSͷ૬ੑ͕ѱ͍͔Βํͳ͖͘߹͍ͬͯΔ DynamoDB͖Ͱ͔͢ʁ
ํ͕ͳ͍͔ΒDynamoDB…… ຊRDBΛ͍͍͚ͨͲ
RDBʹ͑ͯΔΜ͡Όͳ͍Ͱ͔͢ʁʁ NoSQL͕͑ͳ͍Μ͡Όͳͯ͘
RDBͷ͜ͱͪΌΜͱ͔ͬͯ·͔͢ʁʁ ͦͦ
• ίωΫγϣϯϞσϧ • Ӭଓత • ίωΫγϣϯϓʔϦϯά • ηΩϡϦςΟ • ΈࠐΈͷऑ͍ೝূ
• VPCͷΞΫηείετʢENIੜॲཧʣ LambdaͱRDSͷ૬ੑ͕Կނѱ͍ͷ͔આ໌Ͱ͖·͔͢ʁ ͱͯOLTPͰ͑ͳ͍(ඵʙेඵ)
DynamoDBͷ͜ͱ͔ͬͯ·͔͢ʁʁ ͡Ό͋
σʔλϕʔεͷ͜ͱ͔ͬͯ·͔͢ʁʁ ͱ͍͏͔
• ςʔϒϧઃܭͱ͔ΠϯσοΫεઃܭͬͯͲ͏Δͷʁ • ACIDτϥϯβΫγϣϯແ͍ͷʹେৎͳͷʁ • ݁Ռ߹ੑͬͯେৎͳͷʁ • ্ख͘ੑೳΛҾ͖ग़͢ʹͲ͏͢Εྑ͍ʁ DynamoDBͷΑ͋͘Δෆ҆
• ςʔϒϧઃܭͱ͔ΠϯσοΫεઃܭͬͯͲ͏Δͷʁ • ACIDτϥϯβΫγϣϯແ͍ͷʹେৎͳͷʁ • ݁Ռ߹ੑͬͯେৎͳͷʁ • ্ख͘ੑೳΛҾ͖ग़͢ʹͲ͏͢Εྑ͍ʁ DynamoDBͷΑ͋͘Δෆ҆ ͔Βͳ͍͔Βෆ҆ʹͳΔ
• AWS͕ఏڙ͢ΔNoSQLαʔϏε • KeyʹΑΔશҰகͱIndex୳ࡧ͕Մೳ • ϑϧϚωʔδυ • ແ੍ݶͷࣗಈ֦ுετϨʔδʢ͚ͬͨͩʣ • ΠϯελϯεͷཧෆཁɺWrite/Readͷੑೳ͚ͩΛࢦఆ͢Δ
• ͰɺΩϟύγςΟͷࣄલ༧͕ඞཁͳαʔϏεServerless͡Όͳ͍ͬͯCNCF͞Μ͕ݴͬɾɾɾ͋ɺ୭͔དྷͨΈ͍ͨ DynamoDBͷجຊ
DynamoDBͲΜͳ ”AWSαʔϏε” ͔ Ͱͳ͘
DynamoDBͲΜͳ ”σʔλϕʔε” ͔ ͬͯ·͔͢ʁ
• Consistent Hashing • Vector clocks with reconciliation during reads
• Sloppy Quorum and hinted handoff • Anti-entropy using Merkle trees • Gossip-based membership protocol and failure detection Amazon’s Dynamo ৄ͘͠Γ͍ͨͳΒʢBetterͰ͋ͬͯMustͰͳ͍ʣ https://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
• ΩʔͷϋογϡͰࢄ → Partition Key • ͦͷࢄͨ͠தͰIndex͕ߏங͞Ε͍ͯΔ → Sort Key
• όοΫάϥϯυͰߋ৽ΛޙউͪͰಉظ • ݁Ռ߹ੑϕʔε • ॻ͖ࠐΈɾಡΈࠐΈͷQuorumͰ߹ੑ͕ܾ·Δ • ΩʔΛࢦఆͨ͠ಡΈࠐΈڧ߹ੑ͕બՄೳ ͑Δ͖ϙΠϯτ
Partition KeyͷϋογϡͰࢄ ͋ɺ͜ΕRDB(Hash Index)Ͱʢ͈́
• Partition Keyʹ࿈൪Λ͏ͷԿͷҙຯͳ͍͜ͱ͕͔Δ • ࿈൪ΧϯλʔΛ࡞ΔͱPartition͕ภΔ • Max + 1ͳΜͯҎͯͷ֎ •
ཧతʹҰҙͱͳΔଐੑ͕͋ΔͳΒͦΕΛ͏ • ͳ͚ΕUUIDͷΑ͏ͳিಥՄೳੑ͕ۃখͳϥϯμϜͰྑ͍ • SortͰ͖ͳ͍ͷͰSnowflakeͷΑ͏ͳॱংੑཁΒͳ͍ • User IDCognitoͰೝূ͍ͯ͠ΔͳΒCognito͕ൃߦ͢ΔIDΛ͏ • ΞΫηε੍ޚʹ͑Δ ͜ͷલఏΛ౿·͑Δͱ
• 3000RCU, 1000WCU, Index Size 10GB (2018-10-31ݱࡏ) • ͲΕ͔ҰͭͰա͢ΔͱPartitionׂ͕͞ΕΔ •
ׂ͞ΕΔͱ֬อͨ͠Ϣχοτ͕ۉʹׂΓৼΒΕΔ • εϩοτϦϯά࣌ͷҰ࣌తͳόʔετ͕͋Δ • ݶఆతͳόʔετ • ༨Λճ͢ (Adaptive Capacity) Partition͝ͱͷΩϟύγςΟ
Partition KeyͰࢄɺSort Keyʁ
B+TREE INDEX B-treeͳͷ͔B+treeͳͷ͔ެද͞Ε͍ͯͳ͍͚Ͳ Range Keyͱ͔ݺͿ͘Β͍͔ͩΒ…Ͷʁ ͋ɺ͜ΕRDBͰʢ͈́
TARGETING QUERY B+treeͷߏΛ͍ͬͯΕ Έ߹ΘͤΩʔͷΠϯσοΫε͕ແͯ͘ ͜͏Εྑ͍͜ͱ͕͔Δ
Partition KeyͰࢄͨ͠தͰSort KeyͰྻ ͜ͷΠϝʔδΛ࣋ͭ͜ͱ͕ͱͯॏཁ
• Partition KeyͰࢄͨ͠ΠϯελϯεʹIndex͕ߏங͞ΕΔ • ҰͭͷΠϯελϯεͰಉظతʹIndexߋ৽͞ΕΔ • ڧ߹ੑ͕બՄೳ • Primary Sort
KeyͷϦʔϑϊʔυʹ࣮σʔλͷΞυϨε͕ ֨ೲ͞Ε͍ͯΔΠϝʔδʢͱࢥΘΕΔʣ • LSI࣮σʔλΛ࣋ͨͳ͍ͷͰॏෳՄೳ Primary Sort Key, LSI
GSI࣮ςʔϒϧͷࣹӨ
• ू߹ʹ͓͚ΔࣹӨ • GSIͷPartition KeyΛݩʹࢄ ʢߋ৽ඇಉظͰద༻ʣ • ϦʔϑϊʔυʹଐੑͱPrimary Key ʢৄ͘͠ޙड़ʣ
• ιʔτࡁΈͷσʔλͷ෦ෳ ʢҟೝΊΔʣ ࣹӨ ࡶͳMySQLύϑΥʔϚϯενϡʔχϯά from yoku0825 https://www.slideshare.net/yoku0825/mysql-57449062/62
• KEYS_ONLY: Primary KeyͷͷΈ • INCLUDE: Primary Keyͷ + બͨ͠ଐੑ
• ALL: શͯͷଐੑ • GSIʹؚΊͨଐੑPrimary KeyΛҾ͖͢ඞཁ͕ͳ͘ͳΔ • ಡΈࠐΈͷޮԽ • ߋ৽ίετͱͷτϨʔυΦϑ • MySQLPostgreSQLͰݴ͏Covering Index / Index Only ScanͷΠϝʔδ GSI͕ࣹӨ͢ΔଐੑͷΦϓγϣϯ
ࣹӨͱ͍͏͜ͱ 1:Nͷ͕ؔ͜ͷΑ͏ʹղܾͰ͖Δ
• ϝΠϯςʔϒϧͱผͷPartitionͰࢄ͞ΕΔ • ߋ৽ඇಉظͰө • ݁Ռ߹ੑͬͯେৎͳͷʁ • RDSͩͬͯRead ReplicaಡΜͩΒ݁Ռ߹ੑͰ͢Αʁʁ •
GSIͷΫΤϦ͍͍ͩͨRead Replicaʹ͛ΔΑ͏ͳΫΤϦ GSIʹର͢ΔΫΤϦ݁Ռ߹ੑ
͋ɺ͜ΕRDBͰʢԿճʁ
ΈΛΕ࣮ʹγϯϓϧ શ෦RDBͰطʹ͋ͬͨΈͰ͢Αʁʁ
☓ DynamoDBบ͕͋Δ ◦ RDBͷบ͕͋Γ͗͢Δ →γϯϓϧ͗͢ΔDynamoDBͱͷΪϟοϓ͕ڧͯͦ͘͏ײ͡Δ͚ͩ
RDBʹચ͞ΕͯΔΜ͡Όͳ͍Ͱ͔͢ʁʁ ͱ͍͏͔ͦͷRDBͷ͜ͱ͢ΒͪΌΜͱ͔ͬͯ͋͛ͯ·͔ͨ͠ʁ
RDBͷ͜ͱ͔ͬͯ·͔ͨ͠ʁ ͱ͍͏͔ɺͦͷେ͖ͳ
DynamoDBͷσʔλϞσϦϯά ͰͦΕΛ౿·͑ͯ
ͦΕσʔλͷஔΛܾΊΔ͜ͱ ҟೝΊΔʂʂ
σʔλϞσϦϯάͷߟ͑ํͷϙΠϯτ ஔΛܾΊΔͨΊͷ
• εΩʔϚϨεͷҙຯΛߟ͑Δ • List, Map͕࣋ͯΔ͜ͱʁ • ޙ͔ΒଐੑΛγʔϜϨεʹՃͰ͖Δ͜ͱʁ • A. ͳΜͰಥͬࠐΊΔ͜ͱ
• Partition KeyͰࢄ • ςʔϒϧΛ͚ͳͯ͘ෛՙࢄ͢Δ • ۃ͢Δͱ͚Δҙຯ͕ͳ͍ɺ͚ͳ͍ํ͕ΩϟύγςΟ͕ཧ͍͢͠ • “ઃܭ͕༏ΕͨΞϓϦέʔγϣϯͰɺඞཁͳςʔϒϧ 1 ͭͷΈͰ͢” • GSIͷݸ੍ݶ(5ݸ)ͳͲ͋ΔͷͰۀυϝΠϯຖʹҰͭ͘Β͍ͷؾ࣋ͪ ςʔϒϧͷ͚ํ
• RDB • ·ͣεΩʔϚΛߟ͑Δ • ਖ਼نԽ • ͦΕʹରͯ͠Ͳ͏ΞΫηε͢Δ͔(SQL)ߟ͑Δ • DynamoDB
• εΩʔϚϨε • ඇਖ਼نԽ • ·ͣΞΫηεΛߟ͑ͯɺͦΕʹ߹ΘͤͨσʔλΛ࡞Δ • ΞϓϦέʔγϣϯͱσʔλϞσϧಉ࣌ʹઃܭ͢Δ RDBͱΞϓϩʔν͕શ͘ҟͳΔ
ACIDτϥϯβΫγϣϯ͕ແ͍ͷͲ͏͢Δͷʁ ͱ͍͑
• ݪࢠੑ (Atomicity) • Ұ؏(߹)ੑ (Consistency) • ಠཱੑ (Isolation) •
Ӭଓੑ (Durability) ͦͦACIDͬͯͳΜ͚ͩͬʁ ͜ΕΒΛ୲อ͢Δ͜ͱ͕తͰ͋ͬͯ τϥϯβΫγϣϯγεςϜͰ͋Δඞཁແ͍
• RDB • ਖ਼نԽ͍ͯ͘͠ͱςʔϒϧͷجຊతʹ૿͑Δ • ཧతʹ͔Εͨςʔϒϧ(Ϩίʔυ)ؒͷ߹ੑΛऔΔΈ͕͋Δ • ϦϨʔγϣϯγοϓ • ֎෦Ωʔ੍
• ACIDτϥϯβΫγϣϯ • DynamoDB • ඇਖ਼نԽͯ͠1ͭͷ࣮ମʹ͍ͭͯ1ͭͷΞΠςϜʹऩΊΔ • 1ΞΠςϜͷߋ৽ΞτϛοΫ • ͖݅ॻ͖ࠐΈΛָͬͨ؍తϩοΫ ACIDͷ୲อ
• PutItem • ΞΠςϜΛ·Δ͝ͱߋ৽͢Δɺແ͔ͬͨΒ࡞Δ • Get -> ॻ͖͑ -> Putͩͱ߹ੑ่͕ΕΔʢՄೳੑ͕͋Δʣ
• UpdateItem • ΞΠςϜΛ෦ߋ৽ɺແ͔ͬͨΒ࡞Δ • List, Mapͷ෦ՃɺআͳͲՄೳ • ଐੑͷΛར༻ͨ͠ߋ৽Մೳʢ͋Δଐੑʹ+1͢Δ) • ଐੑͷΛར༻͖ͨ݅͠ॻ͖ࠐΈ DynamoDBͷߋ৽ૢ࡞
ඇਖ਼نԽ͢ΔͱಡΈࠐΈ͕ͭΒ͍ ϑϧεΩϟϯ͢Δ͔͠ͳ͍ શ෦GSIషΔͷʁ
CQRSͰ͢Α ͦ͜Ͱ
• ॻ͖ࠐΉσʔλͱಡΈࠐΉσʔλಉ͡Ͱ͋Δඞཁແ͍ • ݁Ռ߹ੑΛड͚ೖΕͯඇಉظͰಡΈࠐΈσʔλΛ࡞ΕΔ • CommandͷྃΛͬͯQuery༻σʔλΛ࡞ΕΔ • DynamoDB Streams͕Ϛον͢Δ •
CQRSΛૉʹ࣮͢Δͱ·ͣKinesis Streams͚ͩͲ • Materialized View CQRS (Command Query Responsibility Segregation)
ARCHITECTURE PATTERN AppSync͕͜Ε͔ΒॏཁͳҐஔΛΊΔͣ
DATA ATOMICITY MicroserviesతͳΠϝʔδͰ͜ͷݪ୯Ґ͕ॏཁ σʔλετΞɾςʔϒϧͷ͚ํͰͳ͍ ލ͕͍ͤͨͳΒMaterialized ViewΛ࡞Δ User Profile User Status
Name Email Read/Write Search Read/Write
͜ΜͳΠϝʔδ 1ͭͷGSIͰ༷ʑͳݕࡧ͕Ͱ͖Δ
• ෳͷ࣮ମʹލΔτϥϯβΫγϣϯ • DynamoDB୯ମͩͱ͜͜Ͳ͏ͯͭ͠Β͍ • τϥϯβΫγϣϯϨϕϧͷ࠷ߴSelializable • ͭ·ΓྻԽ • Kinesis
StreamsͰྻԽ • ίέΔ͔͠Εͳ͍ͷͰႈʹ࣮͢Δ • ݅ॻ͖ࠐΈͰطʹߦͬͨॲཧΛSkip͢Δͷႈ ͱ͍͑ݶք͋Δ
• Kinesis StreamsͰಉ࣌ଓΛίϯτϩʔϧ͢Ε ผʹRDSͬͯԿͷͳ͍Θ͚Ͱ • ಉ࣌ଓ → ShardͱಉͰݻఆ • LambdaجຊWarm
Start • ͭ·ΓɺENIͷԆͱ΄΅ແԑ • σϓϩΠShardม͑ͨλΠϛϯάͰى͖͏Δ͚ͲͦͦඇಉظͳͷͰʢ͈́ ͱ͍͏͔
ADVANCED TIPS ͕࣌ؒ͋Ε͍͔ͭ͘͝հ
• DynamoDBͷIndexSparse • Indexଐੑ͕ଘࡏ͠ͳ͍߹ɺIndexʹͦͷσʔλؚ·ͳ͍ • ಛఆͷΞΠςϜͷΈʹଘࡏ͢ΔଐੑΛࢦఆ͢Δ͜ͱͰ Index༰ྔ͕େ͖͘ѹॖͰ͖Δ Sparse Index
SortཁΒͳ͍ GSIʹ҉ͰPrimary Keyؚ͕·ΕΔ
• Α͋͘Δ֊ߏΛ࣋ͭσʔλ • ৫ߏ • ωετͨ͠ΧςΰϥΠζ • ͳͲͳͲ • ݕࡧΛߟ͑ͳ͚Ε֊ߏΛ·Δͬͱ1ΞΠςϜʹԡ͠ࠐΉ
• Sort KeyͰ֊Λදݱ͢Δ Hierarchical data
Sort KeyͰલํҰகݕࡧ ͋ɺ͜ΕSQLΞϯνύλʔϯͰʢ͈́ ઃܭ͕ࡶͳͷ͝Ѫᇷ
• Itemʹ”Version”ͷΑ͏ͳଐੑΛ࣋ͨͤΔ • ଐੑͷΛར༻͖ͨ݅͠ॻ͖ࠐΈ • ConditionExpression=“Version = 1” • UpdateExpression=“ADD
Version 1” • Version = 1Ͱͳ͚ΕΤϥʔ͕ฦΔ • Version = 1ͳΒ2ʹߋ৽͞ΕΔ όʔδϣχϯάͱָ؍తϩοΫ
• తʹݴͬͯόΠϒϧ • ͜ΕΛಡΜͰશͯཧղͰ͖ͨΒҰਓલʂʂ • https://docs.aws.amazon.com/ja_jp/amazondynamodb/latest/ developerguide/best-practices.html DynamoDB Best Practices
“DynamoDB ދͷר” [ݕࡧ] http://marcy.hatenablog.com/entry/2018/07/31/213705 ҰԠະͳͷͰ·ͩॻ͔͘
“LET’S BECOME THE FRIEND OF RDB” THANKS!! “LET’S BECOME THE
FRIEND OF DYNAMODB” “LET’S BECOME THE FRIEND OF SERVERLESS”