Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Aerospike で 大量のトランザクション処理をさばく
Search
ikosin
October 30, 2015
Programming
8
2.3k
Aerospike で 大量のトランザクション処理をさばく
https://atnd.org/events/71301
での発表資料です。
ikosin
October 30, 2015
Tweet
Share
Other Decks in Programming
See All in Programming
Regular Expressions, REXML, Automata Learning
makenowjust
0
220
A New Era of Testing
mannodermaus
2
490
【TID2024】模擬講義:プログラマと一緒にゲームをデザインしてみよう!
akatsukigames_tech
0
640
Scala におけるコンパイラエラーとの付き合い方
chencmd
2
420
Developer Joy == Developer Productivity (really!)
hollycummins
1
180
rbs-inlineを導入してYARDからRBSに移行する
euglena1215
1
270
Rubyとクリエイティブコーディングの輪の広がり / The Growing Circle of Ruby and Creative Coding
chobishiba
1
270
私のEbitengineの第一歩
qt_luigi
0
450
watsonx.ai Dojo #2 生成AIを使ったアプリ開発入門編
oniak3ibm
PRO
0
110
Method Swizzlingを行うライブラリにおけるマルチモジュール設計
yoshikma
0
120
connect-go で面倒くささと戦う / 2024-08-27 #newmo_layerx_go
izumin5210
2
640
Rubyのobject_id
qnighy
6
1.3k
Featured
See All Featured
Adopting Sorbet at Scale
ufuk
73
8.9k
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
Building Better People: How to give real-time feedback that sticks.
wjessup
359
19k
Visualization
eitanlees
142
15k
Designing on Purpose - Digital PM Summit 2013
jponch
114
6.8k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
26
1.3k
We Have a Design System, Now What?
morganepeng
48
7.1k
How to Ace a Technical Interview
jacobian
274
23k
Six Lessons from altMBA
skipperchong
26
3.3k
Rails Girls Zürich Keynote
gr2m
93
13k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
278
13k
Writing Fast Ruby
sferik
623
60k
Transcript
Aerospike Ͱ େྔͷτϥϯβΫγϣϯॲཧΛ ͘͞
ʘ͜Μʹͪʗʘ͜Μʹͪʗ • @62jun26 • Java Ͱ͢ • ࠷ۙΠϯϑϥدΓͷࣄ͕ଟ͍ • ΠϯλʔωοτͱΧʔϓ͕͖Ͱ͢
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
ΞυςΫྖҬʹ͓͚Δ τϥϯβΫγϣϯܥॲཧ • Ϣʔβใর߹ • ଐੑσʔλΛ༻͍ͨλʔήςΟϯάʹ༻͍Δ • ࠂใऔಘ • ΫϦΤΠςΟϒͷࠂใ
• ࠂ৴࣮ੵΈ্͛ • ༧ࢉཧͷͨΊͷ৴࣮
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
τϥϯβΫγϣϯܥॲཧ ʹඞཁͳཁ݅ • ϨΠςϯγ • ߴεϧʔϓοτ • ߴՄ༻ੑ
͋Δͱخ͍͠ • εέʔϥϏϦςΟ • ؆୯ͳूܭΛ࣮ݱ͢Δ UDF ·ͨΈࠐΈؔ • σʔλͷӬଓԽ =>
RDB Ͱͳ͘σʔλΛγϯϓϧʹͯ͠ KVS Λ ༻͢ΔྲྀΕ
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
͜Ε·ͰͷาΈ • Tokyo Tyrant & Tokyo Cabinet • Redis •
DynamoDB
Tokyo Tyrant & Tokyo Cabinet • ࠃ࢈ KVS • 2010
લޙ • memcache ޓϓϩτίϧͰσʔλӬଓԽՄೳ • ϚελɾεϨʔϒܕͷϨϓϦέʔγϣϯػೳ • ϚϧνϚελߏՄೳ • ·ͩ·ͩݱͰಈ͍ͯΔͱ͍͏ฉ͘
Redis • σʔλӬଓԽՄೳ • ༷ʑͳσʔλߏͷ··อଘՄೳʢจࣈྻ, List, Set, ιʔτࡁΈSet, Hashʣ •
ϚελɾεϨʔϒܕͷϨϓϦέʔγϣϯػೳ • Amazon ͷ ElastiCache ͰϑϧϚωʔδυͳαʔϏεͱͯ͠ར༻Մೳ • 20154݄ʹग़ͨ Redis 3.0 Ͱʹ Redis Cluster ͕ਖ਼ࣜϦϦʔεʂ
༨ஊ: Redis • Ϧιʔε͕๛Ͱ֤ΞϓϦέʔγϣϯαʔό ʹಉࠝͰ͖ΔͳΒ Redis ͕ྑ͍ͱࢥ͍·͢ • Write ϚελʢElastiCache
ʣɺRead ΞϓϦέʔγϣϯαʔόಉࠝͷεϨʔϒ͔ Β
DynamoDB • Amazon ͷϑϧϚωʔδυͳNoSQL σʔλϕʔε • ϨΠςϯγ • ߴεϧʔϓοτ •
ߴՄ༻ੑ • ߴ͍εέʔϥϏϦςΟ • ͓ۚΛ͑શͯͷղܾ͞ΕΔʂ
ʢ͓·͚ʣ ͦͷଞީิʹ͕͋ΔϓϩμΫτ • Memcached • MongoDB • Cassandra • HBase
• BigTable
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
Aerospike IUUQXXXBFSPTQJLFDPNBCPVUBFSPTQJLF
Aerospike ͷಛ • ૣ͍ʂʢϨΠςϯγʣ • ͍҆ʂʢߴεϧʔϓοτ => αʔόઅʣ • ࢫ͍ʂʢࣗಈΫϥελߏɾγϟʔσΟϯάʣ
• ҆৺ʂʢσʔλԽʣ ϕλͰ͢Ͷ
ϨΠςϯγɾߴεϧʔϓοτ • ҎલɺAerospike ͷதͷਓͷهࣄΛ༁͠·ͨ͠ • Aerospike on Amazon EC2 Ͱ
100ສTPS Λͨͬͨ 1.68 υϧ /࣌ Ͱ࣮ݱ͢Δํ๏ https://fa-works.com/blog/1-aerospike-server-x-1-amazon- ec2-instance-achieve-1-million-tps • C3.8xlarge 1Ͱ 100ສTPS ͷ Read ϦΫΤετΛॲཧ • 97% Ҏ্ͷϦΫΤετΛ 1ms Ҏʹฦ٫ʢຊՔಇதͷαʔόʣ
ࣗಈΫϥελߏɾγϟʔσΟ ϯά • ࠾༻ͷܾΊखʹͳͬͨ • ৽͍͠αʔόΛ༻ҙͯ͠ɺطଘϊʔυͷ IP Λ ࢦఆͯ͠ىಈ͢ΕউखʹΫϥελϦϯά •
ύʔςΟγϣϯͷ࠶ܭࢉΛߦͬͯϚΠάϨʔ τʢσʔλϦόϥϯγϯάʣͯ͘͠ΕΔ
σʔλԽ • ϨϓϦέʔγϣϯɾϑΝΫλ (replication factor) ͱ͍͏ ύϥϝʔλͰσʔλͷෳͷΛࢦఆͰ͖Δ • replication factor
= 2 ͷ߹ɺϝΠϯͰΞΫηε͢ΔϚ ελσʔλͱผʹɺଞͷαʔόʹ 1ͭϨϓϦΧσʔλ ͕ଘࡏ͢Δ • 1 Ϋϥογϡͯ͠ɺϨϓϦΧͷσʔλ͕ੜ͖͍ͯΔ ͨΊσʔλΛϩετ͠ͳ͍
্࣮Ͱҙ͢Δ • Limit • Setʢςʔϒϧʣͷ => 1024ݸҎ • BinʢΧϥϜʣ໊ =>
14จࣈҎ • Record => ʢσϑΥϧτʣ128KB ʢ࠷େ1MBʣ • http://www.aerospike.com/docs/guide/FAQ.html
্࣮Ͱҙ͢Δ̎ • ClientLibrary ͰͷγϦΞϥΠζ͕݁ߏඍົ • SecondaryIndex ͰͷෳΠϯσοΫεͷ And ݕࡧෆՄ •
SecondaryIndexͰɺIn ͕۟ͳ͍ • Range αϙʔτ͞Ε͍ͯΔ͕ɺจࣈྻΛҾֻ͚ͬ ʹ͍͘
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
མͱ݀̍͠:κϯϏϨίʔυ Delete & TTL ʹΑΔ Expire আͰͳ͍ʂʂ • ϝϞϦʹࡌͤͯ͋ΔΠϯσοΫεʢPKʣΛআ͢Δ͚ͩ ͰɺετϨʔδ͔Βଈ࠲ʹফ͑ΔΘ͚Ͱͳ͍
• SSD ͷಛੑ্ɺσʔλআෛՙ͕ߴ͍ • σϑϥάͷ͍ͭͰʹ্ॻ͖ʢ͍ͭফ͑Δ͔ෆ໌ʣ • ௨ৗΞΫηεग़དྷͳ͍ͷͰͳ͍ͣͩͬͨ...
མͱ݀̍͠:κϯϏϨίʔυ ͨͩ͠ɺAerospike Λ࠶ىಈͨ͠Β෮׆͢Δʂ • Aerospike ىಈ࣌ʹɺετϨʔδ͔Β PK ΛಡΜͰϝϞϦ ʹΠϯσοΫεΛߏங͢Δ •
আ͞ΕͨϨίʔυͱ͞Εͯͳ͍Ϩίʔυͷ۠ผ͕ͳ͍ʂ • আͨ͠ϨίʔυҰॹʹΠϯσοΫε͞ΕͯಡΈऔΓͰ ͖ΔΑ͏ʹͳΔʂʂʁ
མͱ݀̍͠:κϯϏϨίʔυ • Aerospike ࠶ىಈ࣌ʹɺετϨʔδΛۭʹ͢Δ cold-start-empty ΦϓγϣϯΛ༗ޮʹ͠Α͏ • σʔλ͕Խͯ͋͠Εɺଞͷϊʔυ͔Βσʔ λΛΒ͑Δ
མͱ݀̎͠: ར༻Ͱ͖Δσʔλ༰ྔ • લड़ͷσϑϥάͷͨΊɺ༻ҙͨ͠ετϨʔδɾ ϝϞϦΛશ͍͖ͯΕΔΘ͚Ͱͳ͍ • σϑΥϧτͰετϨʔδͷ 50%, ϝϞϦͷ 60%
͠ ͔ར༻Ͱ͖ͳ͍(high water mark) • 90% Λաͨ͠Βॻ͖ࠐΈ͕Ͱ͖ͳ͍(stop write)
མͱ݀̏͠: ϨίʔυαΠζʹؾΛ͚ͭΔ • વͳ͕Βɺ̍Ϩίʔυ͋ͨΓͷαΠζ͕େ ͖͚ΕωοτϫʔΫଳҬΛѹഭ͢Δ • ͋Ε͜Ε̍Ϩίʔυʹ٧ΊࠐΈա͗ͳ͍ • ৗʹσʔλߏΛݟ͢ϚΠϯυ͕ॏཁ
͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ ࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
͓·͚ɿEC2 Ͱӡ༻͢Δ߹ͷ ύϑΥʔϚϯεʹ͍ͭͯ ओͳϘτϧωοΫ • ωοτϫʔΫ • Disk IO (SSD)
ωοτϫʔΫ • Amazon ͷΧλϩάεϖοΫ 10G Ҏ֎ Low ͱ͔ High ͱ͔ϘΧͨ͠
• ಉ͡ High ͰΠϯελϯελΠϓʹΑ͔ͬͯͳΓҟͳΔ • iperf Ͱܭଌͨ͠ํ͕͍͍ʢެ։ͯ͘͠Ε͍ͯΔਓ͍Δʣ • http://qiita.com/saitara/items/882a3e70639637e24a4f • ྫʣc4.2xlarge => 1.01 Gbits/sec • ྫʣc4.4xlarge => 4.27 Gbits/sec
Disk IO & εϧʔϓοτ • EBS ͗͢Δ • ඞཁͳ IO
ੑೳΛग़͢ͷ Provisioned IOPS Λར༻͢Δ ͜ͱͰ࣮ݱͰ͖Δ͕ɺߴ͍ʂ • ΠϯελϯεετϨʔδ Λ͏ • ཧతʹܨ͕͍ͬͯΔετϨʔδͳͷͰૣ͍ʂ • ͨͩ͠ɺΠϯελϯεͷ stop/start Ͱσʔλشൃ͢Δ