Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
(Advanced) Redis
Search
Jan De Poorter
February 12, 2013
Programming
8
390
(Advanced) Redis
Everything we learned at a client project using Redis - Talk given at #ArrrrUG
Jan De Poorter
February 12, 2013
Tweet
Share
Other Decks in Programming
See All in Programming
Prompt Cachingは本当に効果的なのか検証してみた.pdf
ttnyt8701
0
540
Patched fetch did not work
quramy
4
410
GenU導入でCDKに初挑戦し、悪戦苦闘した話
hideg
0
170
LangGraphでのHuman-in-the-Loopの実装
os1ma
3
1.1k
Ruby Parser progress report 2024
yui_knk
2
230
なぜアジャイルがうまくいかないのか?
yum3
1
110
Swiftコードバトル必勝法
toshi0383
0
170
Perl 5 OOP機構30年史 - Perl 5's OOP Mechanism over the past 30 years
moznion
0
400
[DroidKaigi 2024] Android ViewからJetpack Composeへ 〜Jetpack Compose移行のすゝめ〜 / From Android View to Jetpack Compose: A Guide to Migration
syarihu
1
690
ECMAScript、Web標準の型はどう管理されているか / How ECMAScript and Web standards types are maintained
petamoriken
3
390
Our Websites Need a Lifestyle Change, Not a Diet
ryantownsend
0
150
いまから追い上げる、Jetpack Compose トレーニング
nyafunta9858
0
620
Featured
See All Featured
Designing the Hi-DPI Web
ddemaree
278
34k
Product Roadmaps are Hard
iamctodd
PRO
48
10k
WebSockets: Embracing the real-time Web
robhawkes
59
7.3k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
27
7.4k
Build The Right Thing And Hit Your Dates
maggiecrowley
30
2.3k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
166
48k
Mobile First: as difficult as doing things right
swwweet
221
8.8k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
38
9.2k
Teambox: Starting and Learning
jrom
131
8.7k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
29
2.6k
A better future with KSS
kneath
235
17k
Ruby is Unlike a Banana
tanoku
96
11k
Transcript
(advanced) Redis Advanced is between brackets because we don’t claim
to be advanced users, we just have gained a lot of experience
Jan De Poorter @defv
Redis?
NoSQL
:-D
No, but srsly
Redis is an in-memory key-value data-structure server
in memory with “best effort” persistence
Data structure server Strings SET, SETNX, GET, INCR, DECR, APPEND,
... Hashes HSET, HGET, HGETALL, HDEL, HLEN, ... Lists LPUSH, LPOP, LINDEX, LLEN, LRANGE, ... Sets & Sorted sets SADD, SPOP, SMEMBERS, SDIFF, ZADD, ...
some libraries Ruby redis-rb (gem) hiredis-rb (gem) Node redis (npm)
hiredis (npm)
Q-Music Our use case was the API for the new
Q-Music website and the iPhone and Android applications.
High volume
Pub-Sub
BG jobs with Sidekiq
Some do’s
Serialize values
2 class QApi::Redis 3 class << self 4 def redis
5 @redis ||= Redis.new(config) 6 end ... 181 def set(key, data) 182 redis.set key, to_json(data) 183 end 184 185 def get(key) 186 from_json redis.get(key) 187 end 188 216 private 217 def from_json(result) 218 case result 219 when Array 220 result.map { |r| from_json(r) } 221 when Hash 222 result 223 when nil 224 nil 225 else 226 MultiJson.decode(result) 227 end 228 rescue MultiJson::DecodeError 229 result 230 end 231 232 def to_json(data) 233 MultiJson.encode(data) 234 end 262 end 263 end
Save references
# Write LPUSH plays {“id”: 42, ”title”:”99 Bottles of Rum”}
LPUSH plays {“id”: 49, ”title”:”The Drunken Sailor”} LPUSH plays {“id”: 42, ”title”:”99 Bottles of Rum”} # Read LRANGE plays 0 -1 don’t do this
it’s fine with 100 items
but becomes huge with 90 000 items
or 1 000 000 Justin Bieber mentions ;-)
# Write SET tracks:42 {“id”: 42, ”title”:”99 Bottles of Rum”}
SET tracks:49 {“id”: 42, ”title”:” The Drunken Sailor”} LPUSH plays tracks:42 LPUSH plays tracks:49 LPUSH plays tracks:42 # Read LRANGE plays 0 -1 MGET tracks:42 tracks:49 do this instead
or in Ruby 82 def list_from_references(list, options) 83 references =
redis.lrange list, 84 options[:start], options[:stop] 85 if references.any? 86 from_json redis.mget(*references) 87 else 88 [] 89 end 90 end
Some don’t’s
RDB vs AOF
RDB does fork()
doubling memory usage
Our dataset was 4GB
We had ~8GB of memory
No BGSAVE’s for 4 days
:-(
appendonly yes
Slow(er) start-up
Can be optimized
Monitoring
data:~$ redis-cli redis > INFO redis_version:2.4.16 ... connected_clients:152 ... used_memory:2744963744
used_memory_human:2.56G used_memory_peak:6150240632 used_memory_peak_human:5.73G ... changes_since_last_save:9576 last_save_time:1360661315 redis >
PUBLISH & SUBSCRIBE overload
WebSocket clients
1 client == 1 connection == 1 subscription
Works perfect in development
Works perfect in staging
but in production...
1 new event
sent to 3500 connected clients
Takes about 0.3 seconds
Up to 4 messages per second
:-(
Move logic to websocket server
:-(
1 type of msg == 1 subscription
:-)
MULTI / EXEC
> MULTI OK > RENAME bl:1 bld:1 QUEUED > RENAME
bl:2 bld:2 QUEUED > EXEC 1) OK 2) OK
Works great on 1000’s of commands
Not so much on 1 million commands...
Don’t exceed ~10k commands in 1 transaction
master/slave
no library support
DIY
We didn’t
Limited connections
Redis < 2.6
/* Max number of fd supported */ #define AE_SETSIZE (1024*10)
No more then ~10k connections allowed
Redis >= 2.6
The sky is the limit
(actually, the linux max FD is the limit)
Questions?