Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
(Advanced) Redis
Search
Jan De Poorter
February 12, 2013
Programming
8
400
(Advanced) Redis
Everything we learned at a client project using Redis - Talk given at #ArrrrUG
Jan De Poorter
February 12, 2013
Tweet
Share
Other Decks in Programming
See All in Programming
Strands Agents で実現する名刺解析アーキテクチャ
omiya0555
1
110
テスターからテストエンジニアへ ~新米テストエンジニアが歩んだ9ヶ月振り返り~
non0113
2
240
MCPで実現できる、Webサービス利用体験について
syumai
7
2k
MySQL9でベクトルカラム登場!PHP×AWSでのAI/類似検索はこう変わる
suguruooki
1
220
Reactの歴史を振り返る
tutinoko
1
110
0から始めるモジュラーモノリス-クリーンなモノリスを目指して
sushi0120
0
110
MCPを使ってイベントソーシングのAIコーディングを効率化する / Streamlining Event Sourcing AI Coding with MCP
tomohisa
0
180
PHPカンファレンス関西2025 基調講演
sugimotokei
5
940
SwiftでMCPサーバーを作ろう!
giginet
PRO
2
200
AIのメモリー
watany
11
970
SQLアンチパターン第2版 データベースプログラミングで陥りがちな失敗とその対策 / Intro to SQL Antipatterns 2nd
twada
PRO
27
8.1k
TypeScriptでDXを上げろ! Hono編
yusukebe
3
870
Featured
See All Featured
Designing for humans not robots
tammielis
253
25k
The Cost Of JavaScript in 2023
addyosmani
51
8.6k
A Tale of Four Properties
chriscoyier
160
23k
Speed Design
sergeychernyshev
32
1k
For a Future-Friendly Web
brad_frost
179
9.8k
The Pragmatic Product Professional
lauravandoore
35
6.8k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.6k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
How to Ace a Technical Interview
jacobian
278
23k
Adopting Sorbet at Scale
ufuk
77
9.5k
It's Worth the Effort
3n
185
28k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
Transcript
(advanced) Redis Advanced is between brackets because we don’t claim
to be advanced users, we just have gained a lot of experience
Jan De Poorter @defv
Redis?
NoSQL
:-D
No, but srsly
Redis is an in-memory key-value data-structure server
in memory with “best effort” persistence
Data structure server Strings SET, SETNX, GET, INCR, DECR, APPEND,
... Hashes HSET, HGET, HGETALL, HDEL, HLEN, ... Lists LPUSH, LPOP, LINDEX, LLEN, LRANGE, ... Sets & Sorted sets SADD, SPOP, SMEMBERS, SDIFF, ZADD, ...
some libraries Ruby redis-rb (gem) hiredis-rb (gem) Node redis (npm)
hiredis (npm)
Q-Music Our use case was the API for the new
Q-Music website and the iPhone and Android applications.
High volume
Pub-Sub
BG jobs with Sidekiq
Some do’s
Serialize values
2 class QApi::Redis 3 class << self 4 def redis
5 @redis ||= Redis.new(config) 6 end ... 181 def set(key, data) 182 redis.set key, to_json(data) 183 end 184 185 def get(key) 186 from_json redis.get(key) 187 end 188 216 private 217 def from_json(result) 218 case result 219 when Array 220 result.map { |r| from_json(r) } 221 when Hash 222 result 223 when nil 224 nil 225 else 226 MultiJson.decode(result) 227 end 228 rescue MultiJson::DecodeError 229 result 230 end 231 232 def to_json(data) 233 MultiJson.encode(data) 234 end 262 end 263 end
Save references
# Write LPUSH plays {“id”: 42, ”title”:”99 Bottles of Rum”}
LPUSH plays {“id”: 49, ”title”:”The Drunken Sailor”} LPUSH plays {“id”: 42, ”title”:”99 Bottles of Rum”} # Read LRANGE plays 0 -1 don’t do this
it’s fine with 100 items
but becomes huge with 90 000 items
or 1 000 000 Justin Bieber mentions ;-)
# Write SET tracks:42 {“id”: 42, ”title”:”99 Bottles of Rum”}
SET tracks:49 {“id”: 42, ”title”:” The Drunken Sailor”} LPUSH plays tracks:42 LPUSH plays tracks:49 LPUSH plays tracks:42 # Read LRANGE plays 0 -1 MGET tracks:42 tracks:49 do this instead
or in Ruby 82 def list_from_references(list, options) 83 references =
redis.lrange list, 84 options[:start], options[:stop] 85 if references.any? 86 from_json redis.mget(*references) 87 else 88 [] 89 end 90 end
Some don’t’s
RDB vs AOF
RDB does fork()
doubling memory usage
Our dataset was 4GB
We had ~8GB of memory
No BGSAVE’s for 4 days
:-(
appendonly yes
Slow(er) start-up
Can be optimized
Monitoring
data:~$ redis-cli redis > INFO redis_version:2.4.16 ... connected_clients:152 ... used_memory:2744963744
used_memory_human:2.56G used_memory_peak:6150240632 used_memory_peak_human:5.73G ... changes_since_last_save:9576 last_save_time:1360661315 redis >
PUBLISH & SUBSCRIBE overload
WebSocket clients
1 client == 1 connection == 1 subscription
Works perfect in development
Works perfect in staging
but in production...
1 new event
sent to 3500 connected clients
Takes about 0.3 seconds
Up to 4 messages per second
:-(
Move logic to websocket server
:-(
1 type of msg == 1 subscription
:-)
MULTI / EXEC
> MULTI OK > RENAME bl:1 bld:1 QUEUED > RENAME
bl:2 bld:2 QUEUED > EXEC 1) OK 2) OK
Works great on 1000’s of commands
Not so much on 1 million commands...
Don’t exceed ~10k commands in 1 transaction
master/slave
no library support
DIY
We didn’t
Limited connections
Redis < 2.6
/* Max number of fd supported */ #define AE_SETSIZE (1024*10)
No more then ~10k connections allowed
Redis >= 2.6
The sky is the limit
(actually, the linux max FD is the limit)
Questions?