Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Dgraph - A high performance graph database writ...
Search
Yuichi Saito
April 15, 2018
Technology
7
7.2k
Dgraph - A high performance graph database written in pure Go
Go Conference 2018 Spring
Yuichi Saito
April 15, 2018
Tweet
Share
More Decks by Yuichi Saito
See All by Yuichi Saito
失敗から学ぶ - ポストモーテム / Postmotem culture at Wantedly
munisystem
3
35k
Distributed Tracing with OpenCensus at Wantedly, Inc.
munisystem
3
5.1k
OpenCensus による APM の実現と、未来 / Implementing APM with OpenCensus
munisystem
8
6.8k
Effective Health Checking
munisystem
2
710
Learn more about Admission Webhooks
munisystem
1
1.6k
An introduction to monitoring Go Application with OpenCensus
munisystem
1
1.2k
Other Decks in Technology
See All in Technology
【技術書典17】OpenFOAM(自宅で極める流体解析)2次元円柱まわりの流れ
kamakiri1225
0
210
Gradle: The Build System That Loves To Hate You
aurimas
2
140
とあるユーザー企業におけるリスクベースで考えるセキュリティ業務のお話し
4su_para
3
320
スプリントゴールにチームの状態も設定する背景とその効果 / Team state in sprint goals why and impact
kakehashi
2
100
小規模に始めるデータメッシュとデータガバナンスの実践
kimujun
3
590
いまならこう作りたい AWSコンテナ[本格]入門ハンズオン 〜2024年版 ハンズオンの構想〜
horsewin
9
2.1k
急成長中のWINTICKETにおける品質と開発スピードと向き合ったQA戦略と今後の展望 / winticket-autify
cyberagentdevelopers
PRO
1
160
Nix入門パラダイム編
asa1984
2
200
使えそうで使われないCloudHSM
maikamibayashi
0
170
サイバーエージェントにおける生成AIのリスキリング施策の取り組み / cyber-ai-reskilling
cyberagentdevelopers
PRO
2
200
Java x Spring Boot Warm up
kazu_kichi_67
2
490
「 SharePoint 難しい」ってよく聞くけど、そんなに言うなら8歳の息子に試してもらった
taichinakamura
1
620
Featured
See All Featured
XXLCSS - How to scale CSS and keep your sanity
sugarenia
246
1.3M
The Illustrated Children's Guide to Kubernetes
chrisshort
48
48k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
107
49k
Navigating Team Friction
lara
183
14k
Building Flexible Design Systems
yeseniaperezcruz
327
38k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
131
33k
Fontdeck: Realign not Redesign
paulrobertlloyd
81
5.2k
The Art of Programming - Codeland 2020
erikaheidi
51
13k
GitHub's CSS Performance
jonrohan
1030
460k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
27
4.2k
Building Your Own Lightsaber
phodgson
102
6k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
9
680
Transcript
©2018 Wantedly, Inc. Dgraph - A high performance graph database
written in pure Go Go Conference 2018 Spring - @munisystem
©2018 Wantedly, Inc. Yuichi Saito @munisystem • Wantedly, Inc. •
Infrastructure Engineer • Microservices • Productivity • My Favorite Golang Products • Istio / Istio • wercker / stern • ory / dockertest ࣗݾհ
©2018 Wantedly, Inc. άϥϑͱ άϥϑͰදݱͰ͖Δͷ ࢁͷ8FC"QQMJDBUJPO͕άϥϑΛแ͍ͯ͠Δ w ίʔυϕʔεͱґଘύοέʔδ w υΩϡϝϯτͱΧςΰϦλά
w ͱจཤྺ w ໊ަ
©2018 Wantedly, Inc. ԿނάϥϑσʔλϕʔεΛ͏ͷ͔ طଘͷάϥϑσʔλϕʔεͷ %HSBQIͷհ
·ͱΊ ΞδΣϯμ
©2018 Wantedly, Inc. άϥϑΛ3%#.4Ͱѻ͏ͷ͍͠ w 3%#.4άϥϑΛϦϨʔγϣϯͱͯ͠දݱ͢Δ w άϥϑΛḷΔʹ+0*/Λ͏ඞཁ͕͋Δ w +0*/ίετ͕͔͔Δ
w ਓɺɺ໊ͳͲ༷ʑͳσʔλؒͷάϥϑΛදݱ͢ΔʹɺҎԼͷͲͪΒ͔͔͠ͳ͍ ਓਓɺਓͳͲͦΕͧΕʹରͯ͠ϦϨʔγϣϯΛఆٛ ϙϦϞʔϑΟοΫͳϦϨʔγϣϯΛఆٛ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔
©2018 Wantedly, Inc. άϥϑσʔλϕʔεݸʑͷͭͳ͕ΓϑΥʔΧεͨ͠σʔλετΞ w શͯͷσʔλ/PEFͱ&EHFͰදݱ͞ΕΔ w /PEF w σʔλͷΤϯςΟςΟΛࣔ͢
w FHϢʔβɺɺ໊ w &EHF w σʔλؒͷؔͱํΛࣔ͢ w FH༑ͩͪɺߪೖɺ໊ަ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔ C
©2018 Wantedly, Inc. Index Free Adjacency w /PEF࣍ͷ/PEFΞΫηε͢ΔͨΊͷΞυϨεΛ࣋ͭ w ୳ࡧࣗͷ࣋ͭ&EHFͷ͚ͩͰΑ͍
w -JOLFE-JTUͳͲͰ࣮͞ΕΔ w 0 Ͱ࣍ͷ/PEFΛࢀরͰ͖Δ w ྫ"͔Β#Λࢀর͢Δ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔ A B C
©2018 Wantedly, Inc. Index Free Adjacency w /PEF࣍ͷ/PEFΞΫηε͢ΔͨΊͷΞυϨεΛ࣋ͭ w ୳ࡧࣗͷ࣋ͭ&EHFͷ͚ͩͰΑ͍
w -JOLFE-JTUͳͲͰ࣮͞ΕΔ w 0 Ͱ࣍ͷ/PEFΛࢀরͰ͖Δ w ྫ"͔Β#Λࢀর͢Δ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔ A B C
©2018 Wantedly, Inc. Index Free Adjacency w /PEF࣍ͷ/PEFΞΫηε͢ΔͨΊͷΞυϨεΛ࣋ͭ w ୳ࡧࣗͷ࣋ͭ&EHFͷ͚ͩͰΑ͍
w -JOLFE-JTUͳͲͰ࣮͞ΕΔ w 0 Ͱ࣍ͷ/PEFΛࢀরͰ͖Δ w ྫ"͔Β#Λࢀর͢Δ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔ A B C
©2018 Wantedly, Inc. 8BOUFEMZ1FPQMF͕Ϣʔβʹఏڙ͍ͯ͠ΔՁ w ʮ໊ΛऔΓɺਓͱͭͳ͕Δʯ w ໊ͱਓͷ͍͋ͩͷάϥϑΛετΞ͍ͨ͠ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔
©2018 Wantedly, Inc. طଘͷάϥϑσʔλϕʔεͷ
©2018 Wantedly, Inc. ॻ͖ࠐΈͷੑೳ͕͔ͬͨ w 3FBEຆͲͷάϥϑσʔλϕʔεͰߴ͍ੑೳΛग़͍ͯ͠Δ w ໊ΛࡱΔɺͱ͍͏ߦҝ8BOUFEMZ1FPQMFͷίΞͳମݧ w ຖຖඵେྔʹૹΒΕͯ͘Δ໊ΛετΞͰ͖Δ͚ͩͷ8SJUFͷύϑΥʔϚϯε͕ඞཁ
w ͜ΕΛຬͨ͢άϥϑσʔλϕʔεଘࡏ͠ͳ͔ͬͨ طଘͷάϥϑσʔλϕʔεͷ
©2018 Wantedly, Inc. εέʔϧ͠ͳ͍ߏ • αʔϏεͷʹεέʔϧͰԠ͡ΒΕΔඞཁ͕͋Δ w /FPKσʔλϕʔεଆͰͷσʔλͷγϟʔσΟϯάʹରԠ͍ͯ͠ͳ͍ w 0SJFOU%#$MBTTʢςʔϒϧʣຖʹγϟʔσΟϯάΛ͢Δ
w ͋Δςʔϒϧ͚ͩॻ͖ࠐΈ͕ܹ͍͠ɺΈ͍ͨͳέʔεʹରԠͰ͖ͳ͍ طଘͷάϥϑσʔλϕʔεͷ
©2018 Wantedly, Inc. %HSBQIͷհ
©2018 Wantedly, Inc. (Pͷࢄάϥϑσʔλϕʔε w (SBQI2-ϥΠΫͳΫΤϦγϯλοΫε w H31$ͱ)551ͷαϙʔτ w ϋΠύϑΥʔϚϯε
%HSBQIͱ
©2018 Wantedly, Inc. ॻ͖ࠐΈͷύϑΥʔϚϯε͕༏Ε͍ͯΔ w ެࣜͷϕϯνϚʔΫ w ಡΈॻ͖ڞʹ/FPKͷഒ w 1FPQMFͷ̎ޙΛఆͨ͠σʔλྔ
εϧʔϓοτͰϕϯνϚʔΫΛ࣮ࢪ w εέʔϧͳ͠Ͱ͑ΒΕͨ %HSBQIͱ https://blog.dgraph.io/post/benchmark-
©2018 Wantedly, Inc. %HSBQIͷσʔλϞσϧ w %HSBQIͰάϥϑϞσϧͱͯ͠3%'5SJQMFΛ࠾༻ w 8FCϦιʔεʹϝλใΛ༩͢ΔͨΊͷΈ w ϦιʔεͷؔΛ4VCKFDU
1SFEJDBUF 0CKFDUͰදݱ w 4VCKFDUࢧͱͳΔ/PEF w 1SFEJDBUFؔΛද͢&EHF w 0CKFDUऴͱͳΔ/PEF %HSBQIͱ <a> <is> <person> . <b> <is> <person> . <a> <named> <Alice> . <b> <named> <Bob> . <a> <friend> <b> . RDF ٖࣅίʔυʹ͓͚ΔϑϨϯυϦϨʔγϣϯ
©2018 Wantedly, Inc. 3%'5SJQMF Dgraphͱ <a> <is> <person> . <b>
<is> <person> . <a> <named> <Alice> . <b> <named> <Bob> . <a> <friend> <b> . RDF ٖࣅίʔυʹ͓͚ΔϑϨϯυϦϨʔγϣϯ C Bob b a Alice friend named ==
©2018 Wantedly, Inc. 3%'5SJQMF w QFSTPO͕ͳ͍ w 0CKFDU4VCKFDU&OUJUZͱ"UUSJCVUF͕͋Δ w
&OUJUZ6TFSͳͲͷ֓೦Λද͢ w "UUSJCVUFΛද͢ w ຊ࣭తʹϞσϦϯάʹ͕ؔͳ͍ w OBNFEQSFEJDBUF͕͋ΔͳΒͦΕQFSTPO w ·ΔͰ%VDL5ZQJOH Dgraphͱ C Bob b a Alice friend named
©2018 Wantedly, Inc. %HSBQIͷσʔλߏ
©2018 Wantedly, Inc. άϥϑ̎ͭͷσʔλߏʹΑͬͯදݱ͞Ε͍ͯΔ w 1PTUJOH w 1PTUJOH-JTU %HSBQIͷσʔλߏ
©2018 Wantedly, Inc. 1PTUJOH w %HSBQIͷσʔλͷ࠷খ୯Ґ w 1SFEJDBUFͱͦΕʹඥͮ͘ 4VCKFDUͱ0CKFDUΛแ͢Δ w
WBMVF@UZQF͕6*% w 4VCKFDU&OUJUZͰ͋Δ w ͦΕҎ֎"UUSJCVUF w WBMVFʹ࣮σʔλ͕ೖΔ %HSBQIͷσʔλߏ message Posting { fixed64 uid = 1; bytes value = 2; enum ValType { DEFAULT = 0; BINARY = 1; INT = 2; FLOAT = 3; BOOL = 4; DATETIME = 5; GEO = 6; UID = 7; PASSWORD = 8; STRING = 9; } ValType val_type = 3; ... ... enum PostingType { REF=0; VALUE=1; VALUE_LANG=2; } PostingType posting_type = 4; bytes lang_tag = 5; string label = 6; repeated api.Facet facets = 9; uint32 op = 12; uint64 start_ts = 13; uint64 commit_ts = 14; } Protobuf ʹΑΔ Posting ͷఆٛ
©2018 Wantedly, Inc. 1PTUJOH-JTU w 1SFEJDBUFͱ4VCKFDUͷϖΞʹඥͮ͘1PTUJOHΛͻͱ·ͱΊʹͨ͠ͷ w ͻͱ·ͱΊʹ͢Δ͜ͱͰɺ͋ΔϢʔβͷ༑ୡҰཡ ΛҰճͷ୳ࡧͰऔಘͰ͖Δ w
,74ͷLFZʹ1SFEJDBUFͱ4VCKFDUؚ͕·ΕΔ w ͦͷҝ1PTUJOHʹ͕ࣗԿऀͰ͋Δ͔Λ දݱ͢ΔGJFME͕ͳ͍ %HSBQIͷσʔλߏ message PostingList { repeated Posting postings = 1; bytes checksum = 2; uint64 commit = 3; bytes uids = 4; } Protobuf ʹΑΔ Posting ͷఆٛ (Predicate,Subject) -> PostingList KVS ͷετΞํ๏
©2018 Wantedly, Inc. #BEHFS w %HSBQIͷ։ൃνʔϜ͕࡞ͨ͠(PͰॻ͔Εͨ,74 w ϋΠύϑΥʔϚϯε w 3BOEPN3FBE'BDFCPPLͷ,74Ͱ͋Δ3PDLT%#ͷഒ
w 44%ͷ࠷దԽ w "$*%5SBOTBDUJPOͷαϙʔτ DdgraphͷσʔλετϨʔδ
©2018 Wantedly, Inc. Կނ#BEHFSΛ࡞ͬͨͷ͔ w ͱͱ3PDLT%#Λ͍ͬͯͨ w ͔͠͠$HPΛ͔ͭ͏σϝϦοτ͕େ͖͗ͨ͢ w (PͷΤίγεςϜʹΕͳ͍
w QQSPGSBDFEFUFDUPS w QUISFBE@DSFBUFͰόάΛҾ͖ى͜͠ɺύϑΥʔϚϯε͕Լ͢Δ w (Pͷࠒ w ($͕ޮ͔ͳ͍ͷͰ$GSFF ΛΕΔͱϝϞϦϦʔΫΛҾ͖ى͜͢ DdgraphͷσʔλετϨʔδ
©2018 Wantedly, Inc. Կނ#BEHFSΛ࡞ͬͨͷ͔ w (PͰॻ͔Ε͍ͯΔ#PMU%#Ͳ͏ͩͬͨͷ͔ w #PMU%#3FBE8SJUFڞʹάϩʔόϧۭؒʹ͋ΔTZOD38.VUFYͷϩοΫΛͱΔ w ฒྻॲཧ͕ग़དྷͳ͍
w 1PTUJOH-JTUͷมߋʹେ͖ͳӨڹΛ༩͑ͨ DdgraphͷσʔλετϨʔδ
©2018 Wantedly, Inc. #BEHFSͷΈ
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF w ίϯηϓτͰ͖Δ͚ͩγʔΫΛগͳ͘͢Δ w ॻ͖ࠐΈͷϑϩʔ ॻ͖ࠐΈ͕͋ͬͨ߹8"-ͱϝϞϦʢ.FN5BCMFʣʹॻ͖ग़͢
ॻ͖ࠐΉͱ͖ʹ.FN5BCMFʹσʔλ͕ೖΒͳ͍߹ϑΝΠϧʹϑϥογϡ γʔΫͷྔΛݮΒͨ͢ΊʹɺϑΝΠϧ͕ଟ͘ͳ͖ͬͯͨΒϑΝΠϧΛϚʔδͤ͞Δ w $PNQBDUJPO Storage architecture of Badger
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (muni, 30) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Carol 30 Level 0 (Mem Table)
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (muni, 30) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Carol 30 muni 30 Level 0 (Mem Table)
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (muni, 30) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Level 0 (Mem Table) L W Carol 10 muni 20
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Set: (Alice, 20) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Level 0 (Mem Table) L W Carol 10 muni 20
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Set: (Alice, 20) ... will remove
(Alice, 10) when merge Storage architecture of Badger L W Alice 10 Bob 20 Level 1 L W Alice 20 Level 0 (Mem Table) L W Carol 10 muni 20
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (mike, 50) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Alice 20 Level 0 (Mem Table) L W Carol 10 muni 20
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (mike, 50) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Alice 20 mike 50 Level 0 (Mem Table) L W Carol 10 muni 20
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (mike, 50) Storage architecture of
Badger L W Alice 20 mike 50 Level 1 L W Level 0 (Mem Table) L W Carol 10 muni 20 Bob 20 Level 2
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF w Կ͕خ͍͠ʁ w 8SJUF͕ΔͷϚʔδͷλΠϛϯά͚ͩ w 8"-ॻ͍͍ͯΔ͚Ͳɺ࣮σʔλʹൺΕίετ
w ಡΈࠐΉͱ͖Ϩϕϧ͔Βॱʹߦ͏ͷͰɺஞ࣍σʔλΛߋ৽͢Δඞཁ͕ͳ͍ w ϑΝΠϧͷ͚ͩγʔΫ͢ΕΑ͍ w ϑΝΠϧʹͲͷ,FZؚ͕·Ε͍ͯΔͷ͔֬ೝ͢ΔͨΊͷ#MPPN'JMUFS͕͋Δ w #MPPN'JMUFS͕GBMTFΛฦͨ͠߹ͦͷϑΝΠϧʹ,FZඞͣଘࡏ͠ͳ͍ Storage architecture of Badger
©2018 Wantedly, Inc. "$*%5SBOTBDUJPO w ࣮.VMUJ7FSTJPO$PODVSSFODZ$POUSPM w 3FBEͨ͠ͱ͖ɺߦʹUJNFTUBNQΛ༩͢Δ w ผͷτϥϯβΫγϣϯͰಉ͡ߦʹະίϛοτͷมߋ͕ͬͨ
w มߋϝϞϦʹରͯ͠ߦͳΘΕɺଞͷτϥϯβΫγϣϯ͔Βݟ͑ͳ͍ w ίϛοτͯ͠ߦͷ࣮σʔλͷUJNFTUBNQΛߋ৽ w τϥϯβΫγϣϯͰҰճ3FBEͨ͠ߦΛ͏Ұࢀর w طʹUJNFTUBNQ͕༩͞Ε͍ͯΔͷͰલͷΛฦ͢ ACID Transaction of Badger
©2018 Wantedly, Inc. %HSBQIͷεέʔϦϯά
©2018 Wantedly, Inc. %HSBQIΛߏ͢Δίϯϙʔωϯτ w [FSP w Ϋϥελͷཧϊʔυ w (SPVQʢޙड़ʣͷཧ
w όοΫΞοϓγϟοτμϯͱ໋͍ͬͨྩΛ(SPVQʹൖͤ͞Δ w TFSWFS w σʔλετΞɺΫΤϦͷ࣮ߦϊʔυ w ̍ͭͷ(SPVQʹඞͣଐ͢Δ %HSBQIͷεέʔϦϯά
©2018 Wantedly, Inc. (SPVQ w Ҏ্ͷTFSWFSʹΑͬͯߏ͞ΕΔ w ಉ͡(SPVQʹଐ͢Δશͯͷϊʔυಉ͡σʔλΛ࣋ͭ w 3BGUͷ2VPSVN
w γϟʔσΟϯάͷ୯Ґ w (SPVQʹׂΓͯΒΕΔσʔλ1SFEJDBUFຖʹܾ·Δ w ͭ·ΓGSJFOEʹؔ͢Δσʔλશͯಉ͡(SPVQͰཧ͞ΕΔ w Ͳͷ1SFEJDBUFΛड͚͔࣋ͭ[FSPʹΑܾͬͯఆ͞ΕΔ %HSBQIͷεέʔϦϯά
©2018 Wantedly, Inc. w ͭͳ͕ΓΛอଘ͢ΔσʔλετΞͱͯ͠ άϥϑσʔλϕʔεΛ࠾༻ w %HSBQIطଘͷάϥϑσʔλϕʔε͕ ๊͍͑ͯͨΛղܾ͍ͯ͠Δ w
ॻ͖ࠐΈͷύϑΥʔϚϯε w ਫฏεέʔϦϯά ·ͱΊ