Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless??そんなことよりDBだ!! / We should know Datab...

Masashi Terui
November 03, 2018

Serverless??そんなことよりDBだ!! / We should know Databases before Serverless

JAWS Festa 2018の資料です
https://jft2018.jaws-ug.jp/session/729/

Masashi Terui

November 03, 2018
Tweet

More Decks by Masashi Terui

Other Decks in Technology

Transcript

  1. SERVERWORKS CO.,LTD. + FREELANCER • Serverless Oji-san • Serverless Framework

    Plugin Developer • Serverlessconf Tokyo 2016,2017,2018 speaker • Remote worker (in Sapporo-shi, Hokkaido) • http://marcy.hatenablog.com/entry/2018/08/04/231241 MASASHI TERUI ARCHITECT / DEVELOPER
  2. • ίωΫγϣϯϞσϧ • Ӭଓత • ίωΫγϣϯϓʔϦϯά • ηΩϡϦςΟ • ૊ΈࠐΈͷऑ͍ೝূ

    • VPC಺΁ͷΞΫηείετʢENIੜ੒ॲཧʣ LambdaͱRDSͷ૬ੑ͕Կނѱ͍ͷ͔આ໌Ͱ͖·͔͢ʁ ͱͯ΋OLTPͰ͸࢖͑ͳ͍(਺ඵʙे਺ඵ)
  3. • Consistent Hashing • Vector clocks with reconciliation during reads

    • Sloppy Quorum and hinted handoff • Anti-entropy using Merkle trees • Gossip-based membership protocol and failure detection Amazon’s Dynamo ৄ͘͠஌Γ͍ͨͳΒʢBetterͰ͋ͬͯMustͰ͸ͳ͍ʣ https://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
  4. • Ωʔͷϋογϡ஋Ͱ෼ࢄ → Partition Key • ͦͷ෼ࢄͨ͠தͰIndex͕ߏங͞Ε͍ͯΔ → Sort Key

    • όοΫάϥϯυͰߋ৽ΛޙউͪͰಉظ • ݁Ռ੔߹ੑϕʔε • ॻ͖ࠐΈɾಡΈࠐΈͷQuorum਺Ͱ੔߹ੑ͕ܾ·Δ • ΩʔΛࢦఆͨ͠ಡΈࠐΈ͸ڧ੔߹ੑ͕બ୒Մೳ ཈͑Δ΂͖ϙΠϯτ
  5. • Partition Keyʹ࿈൪Λ࢖͏ͷ͸Կͷҙຯ΋ͳ͍͜ͱ͕෼͔Δ • ࿈൪Χ΢ϯλʔΛ࡞ΔͱPartition͕ภΔ • Max + 1ͳΜͯҎͯͷ֎ •

    ࿦ཧతʹҰҙͱͳΔଐੑ͕͋ΔͳΒͦΕΛ࢖͏ • ͳ͚Ε͹UUIDͷΑ͏ͳিಥՄೳੑ͕ۃখͳϥϯμϜ஋Ͱྑ͍ • Sort΋Ͱ͖ͳ͍ͷͰSnowflakeͷΑ͏ͳॱংੑ΋ཁΒͳ͍ • User ID͸CognitoͰೝূ͍ͯ͠ΔͳΒCognito͕ൃߦ͢ΔIDΛ࢖͏ • ΞΫηε੍ޚʹ΋࢖͑Δ ͜ͷલఏΛ౿·͑Δͱ
  6. • 3000RCU, 1000WCU, Index Size 10GB (2018-10-31ݱࡏ) • ͲΕ͔ҰͭͰ΋௒ա͢ΔͱPartition͕෼ׂ͞ΕΔ •

    ෼ׂ͞ΕΔͱ֬อͨ͠Ϣχοτ͕ۉ౳ʹׂΓৼΒΕΔ • εϩοτϦϯά࣌ͷҰ࣌తͳόʔετ͕͋Δ • ݶఆతͳόʔετ • ༨৒෼Λճ͢ (Adaptive Capacity) Partition͝ͱͷΩϟύγςΟ
  7. • Partition KeyͰ෼ࢄͨ͠ΠϯελϯεʹIndex͕ߏங͞ΕΔ • ҰͭͷΠϯελϯε಺Ͱ͸ಉظతʹIndex΋ߋ৽͞ΕΔ • ڧ੔߹ੑ͕બ୒Մೳ • Primary Sort

    KeyͷϦʔϑϊʔυʹ࣮σʔλͷΞυϨε͕
 ֨ೲ͞Ε͍ͯΔΠϝʔδʢͱࢥΘΕΔʣ • LSI͸࣮σʔλΛ࣋ͨͳ͍ͷͰॏෳՄೳ Primary Sort Key, LSI
  8. • ू߹࿦ʹ͓͚ΔࣹӨ • GSIͷPartition KeyΛݩʹ෼ࢄ
 ʢߋ৽͸ඇಉظͰద༻ʣ • ϦʔϑϊʔυʹଐੑͱPrimary Key
 ʢৄ͘͠͸ޙड़ʣ

    • ιʔτࡁΈͷσʔλͷ෦෼ෳ੡
 ʢҟ࿦͸ೝΊΔʣ ࣹӨ ࡶͳMySQLύϑΥʔϚϯενϡʔχϯά from yoku0825 https://www.slideshare.net/yoku0825/mysql-57449062/62
  9. • KEYS_ONLY: Primary Keyͷ஋ͷΈ • INCLUDE: Primary Keyͷ஋ + બ୒ͨ͠ଐੑ

    • ALL: શͯͷଐੑ • GSIʹؚΊͨଐੑ͸Primary KeyΛҾ͖௚͢ඞཁ͕ͳ͘ͳΔ • ಡΈࠐΈͷޮ཰Խ • ߋ৽ίετͱͷτϨʔυΦϑ • MySQL΍PostgreSQLͰݴ͏Covering Index / Index Only ScanͷΠϝʔδ GSI͕ࣹӨ͢ΔଐੑͷΦϓγϣϯ
  10. • εΩʔϚϨεͷҙຯΛߟ͑Δ • List, Map͕࣋ͯΔ͜ͱʁ • ޙ͔ΒଐੑΛγʔϜϨεʹ௥ՃͰ͖Δ͜ͱʁ • A. ͳΜͰ΋ಥͬࠐΊΔ͜ͱ

    • Partition KeyͰ෼ࢄ • ςʔϒϧΛ෼͚ͳͯ͘΋ෛՙ͸෼ࢄ͢Δ • ۃ࿦͢Δͱ෼͚Δҙຯ͕ͳ͍ɺ෼͚ͳ͍ํ͕ΩϟύγςΟ͕؅ཧ͠΍͍͢ • “ઃܭ͕༏ΕͨΞϓϦέʔγϣϯͰ͸ɺඞཁͳςʔϒϧ͸ 1 ͭͷΈͰ͢” • GSIͷݸ਺੍ݶ(5ݸ)ͳͲ΋͋ΔͷͰۀ຿υϝΠϯຖʹҰͭ͘Β͍ͷؾ࣋ͪ ςʔϒϧͷ෼͚ํ
  11. • RDB • ·ͣεΩʔϚΛߟ͑Δ • ਖ਼نԽ • ͦΕʹରͯ͠Ͳ͏ΞΫηε͢Δ͔(SQL)ߟ͑Δ • DynamoDB

    • εΩʔϚϨε • ඇਖ਼نԽ • ·ͣΞΫηεΛߟ͑ͯɺͦΕʹ߹ΘͤͨσʔλΛ࡞Δ • ΞϓϦέʔγϣϯͱσʔλϞσϧ͸ಉ࣌ʹઃܭ͢Δ RDBͱ͸Ξϓϩʔν͕શ͘ҟͳΔ
  12. • ݪࢠੑ (Atomicity) • Ұ؏(੔߹)ੑ (Consistency) • ಠཱੑ (Isolation) •

    Ӭଓੑ (Durability) ͦ΋ͦ΋ACIDͬͯͳΜ͚ͩͬʁ ͜ΕΒΛ୲อ͢Δ͜ͱ͕໨తͰ͋ͬͯ τϥϯβΫγϣϯγεςϜͰ͋Δඞཁ͸ແ͍
  13. • RDB • ਖ਼نԽ͍ͯ͘͠ͱςʔϒϧͷ਺͸جຊతʹ૿͑Δ • ࿦ཧతʹ෼͔Εͨςʔϒϧ(Ϩίʔυ)ؒͷ੔߹ੑΛऔΔ࢓૊Έ͕͋Δ • ϦϨʔγϣϯγοϓ • ֎෦Ωʔ੍໿

    • ACIDτϥϯβΫγϣϯ • DynamoDB • ඇਖ਼نԽͯ͠1ͭͷ࣮ମʹ͍ͭͯ1ͭͷΞΠςϜʹऩΊΔ • 1ΞΠςϜͷߋ৽͸ΞτϛοΫ • ৚݅෇͖ॻ͖ࠐΈΛ࢖ָͬͨ؍తϩοΫ ACIDͷ୲อ
  14. • PutItem • ΞΠςϜΛ·Δ͝ͱߋ৽͢Δɺແ͔ͬͨΒ࡞Δ • Get -> ॻ͖׵͑ -> Putͩͱ੔߹ੑ่͕ΕΔʢՄೳੑ͕͋Δʣ

    • UpdateItem • ΞΠςϜΛ෦෼ߋ৽ɺແ͔ͬͨΒ࡞Δ • List, Mapͷ෦෼௥Ճɺ࡟আͳͲ΋Մೳ • ଐੑͷ஋Λར༻ͨ͠ߋ৽΋Մೳʢ͋Δଐੑʹ+1͢Δ౳) • ଐੑͷ஋Λར༻ͨ͠৚݅෇͖ॻ͖ࠐΈ DynamoDBͷߋ৽ૢ࡞
  15. • ෳ਺ͷ࣮ମʹލΔτϥϯβΫγϣϯ • DynamoDB୯ମͩͱ͜͜͸Ͳ͏ͯ͠΋ͭΒ͍ • τϥϯβΫγϣϯ෼཭Ϩϕϧͷ࠷ߴ͸Selializable • ͭ·Γ௚ྻԽ • Kinesis

    StreamsͰ௚ྻԽ • ίέΔ͔΋͠Εͳ͍ͷͰႈ౳ʹ࣮૷͢Δ • ৚݅෇ॻ͖ࠐΈͰطʹߦͬͨॲཧΛSkip͢Δͷ΋ႈ౳ ͱ͸͍͑ݶք͸͋Δ
  16. • Kinesis StreamsͰಉ࣌઀ଓ਺Λίϯτϩʔϧ͢Ε͹
 ผʹRDS࢖ͬͯ΋Կͷ໰୊΋ͳ͍Θ͚Ͱ • ಉ࣌઀ଓ਺ → Shard਺ͱಉ਺Ͱݻఆ • Lambda͸جຊWarm

    Start • ͭ·ΓɺENIͷ஗Ԇͱ΄΅ແԑ • σϓϩΠ΍Shard਺ม͑ͨλΠϛϯάͰ͸ى͖͏Δ͚Ͳͦ΋ͦ΋ඇಉظͳͷͰʢ͈́ ͱ͍͏͔
  17. • Itemʹ”Version”ͷΑ͏ͳଐੑΛ࣋ͨͤΔ • ଐੑͷ஋Λར༻ͨ͠৚݅෇͖ॻ͖ࠐΈ • ConditionExpression=“Version = 1” • UpdateExpression=“ADD

    Version 1” • Version = 1Ͱͳ͚Ε͹Τϥʔ͕ฦΔ • Version = 1ͳΒ2ʹߋ৽͞ΕΔ όʔδϣχϯάͱָ؍తϩοΫ
  18. “LET’S BECOME THE FRIEND OF RDB” THANKS!! “LET’S BECOME THE

    FRIEND OF DYNAMODB” “LET’S BECOME THE FRIEND OF SERVERLESS”