Redis: based on real story

Talk @ NoSQL Taiwan

Transcript

1. Poga 2012.03.31 based on real story
2. Hi 你好 Hallo 안녕 ciao salut こんにちは
3. i Poga
4. i wear many hats
5. i wear many hats build products
6. Product released PARTY? Let’s
7. Database CPU Utilization
8. Performance Degrading = Unhappy User = Less Profit
9. Complex Queries Inefficient schema
10. Limited Time
12. Just ADD it to your stack
13. Just ADD it to your stack Not Replace
14. Special Performance Simple In-memory Atomic
15. Special Performance Simple In-memory Atomic
16. Management Problem a.k.a. “We don’t have a MIS”
17. Dependency Feb 25, 2009 First public beta Oct 14, 2011 Redis 2.4 First external dependency jemalloc Mar 31, 2012
18. Installation Feb 25, 2009 First public beta Oct 14, 2011 Redis 2.4 First external dependency jemalloc Mar 31, 2012 make install make install
19. Stable
21. Special Performance Simple In-memory Atomic
22. Special Performance Simple In-memory Atomic
23. In-Memory Memcached High Performance Simple Key-Value Redis In-Memory High Performance Data Structures Persistence Replication ...
24. Data Structures List, Set, Sorted Set, Hash, String http://redis.io/commands
25. Understand your problem http://nosql.mypopescu.com/post/18977451039/redis-guide-what-each-redis-data-type-should-be-used Every command in Redis is manipulating a data structure
26. Special Performance Simple In-memory Atomic
27. Special Performance Simple In-memory Atomic
28. =
29. http://redis.io/topics/benchmarks FAST for real
30. Special Performance Simple In-memory Atomic
31. No Race Condition every command in redis is atomic use redis as a incremental counter “There are very few systems that support an “increment and return” operation, and Redis is one of them (Oracle sequences are another).” http://nateware.com/2010/02/18/an-atomic-rant/
32. No Race Condition every command in redis is atomic Combine multiple command: still atomic! MULTI SET key 123 SET key2 456 EXEC Warning: Don’t confuse with transaction
33. Special Performance Simple In-memory Atomic
34. i
35. Did I make any changes to MySQL? NO It’s still our main database
36. Did I solve the problem? YES solve complex queries via Redis data structure
37. Happy Database Add Redis to stack
38. How? Sorted Set: Per user notification String: API request limiter String, List: Request logging List: Job Queue
39. Real World = Risk Persistence Replication Sharding
40. Persistence RDB AOF
41. RDB • Snapshot • Compact • .rdb file can be transfered to other backup system • Chances to lose data
42. AOF • logs of every write operation • durable • bigger file size
43. You want a degree of safety comparable to PostgreSQL? Just use BOTH
44. Replication Master Slave Slave Slave
45. Replication • Read-only salves • Save on slaves
46. Sharding • Memory is limited • Split data into multiple redis servers
47. Presharding Single Machine Redis instance Redis instance Redis instance Redis instance Redis instance Redis instance Redis instance Redis instance Redis instance Redis instance Multiple Machine
48. Some Case Study • Resque • Rubygems.org • Tumblr • Instagram
49. Resque High Performance Job Queue https://github.com/blog/542-introducing-resque
50. Rubygems.org http://www.scribd.com/doc/34269414/Redis-Persistence-Power feature switch API request limit Logging Job Queue
51. Tumblr http://engineering.tumblr.com/post/7819252942/staircar-redis-powered-notifications Per-user Notification 30,000 request/s per server
52. Instagram http://engineering.tumblr.com/post/7819252942/staircar-redis-powered-notifications Simple Key-value mapping 1/3 cost on hardware
53. Cross server temporary file sharing i Redis string is binary-safe
54. Future
55. Future Lua Scripting Watchdog
56. Lua Scripting Atomic Saving bandwidth between servers
57. Ruby Redis Sum up 200,000 values ... 200,000 times
58. Ruby Redis Sum up 200,000 values, with Lua scripting 1 request
59. Data Model Query Language Scripting MySQL Redis Relational Table SQL Data Structures Lua Lua
60. Watchdog Let Redis monitor itself Like slow queries in MySQL More information to diagnose problem
61. Bright Future Lua Scripting Watchdog
62. = Unique
63. Unique = Not replacing anyone Job security of other database
64. What will u get
65. More life less cost Happy User
66. Thank you question? @devpoga