Intro to Cassandra

Presented at a joint meeting of the Austin Java Users Group and the Austin Python Users Group on March 25, 2014.

Transcript

1. ©2014 DataStax Cassandra committer at DataStax tyler@datastax.com @tylhobbs Tyler Hobbs Intro to Cassandra 1 Tuesday, March 25, 14
2. Five years of Cassandra Jul-09 May-10 Feb-11 Dec-11 Oct-12 Jul-13 0.1 0.3 0.6 0.7 1.0 1.2 ... 2.0 DSE Jul-08 Tuesday, March 25, 14
3. Tuesday, March 25, 14
4. Application/Use Case • Social Signals: like/want/own features for eBay product and item pages • Hunch taste graph for eBay users and items • Many time series use cases Why Cassandra? • Multi-datacenter • Scalable • Write performance • Distributed counters • Hadoop support ACE Tuesday, March 25, 14
5. Time series data Tuesday, March 25, 14
6. Multi-datacenter support Tuesday, March 25, 14
7. Tuesday, March 25, 14
8. Distributed counters Tuesday, March 25, 14
9. Hadoop support Tuesday, March 25, 14
10. Application/Use Case • Adobe AudienceManager: web analytics, content management, and online advertising Why Cassandra? • Low-latency • Scalable • Multi-datacenter • Tuneable consistency ACE Tuesday, March 25, 14
11. Low latency Tuesday, March 25, 14
12. Bootstrapping Tuesday, March 25, 14
13. Bootstrapping Tuesday, March 25, 14
14. Bootstrapping s d s d s d s d Tuesday, March 25, 14
15. Bootstrapping s d s d s d s d Tuesday, March 25, 14
16. Bootstrapping Tuesday, March 25, 14
17. Tuneable consistency •(We’ll come back to this) Tuesday, March 25, 14
18. Application/Use Case • Logging • Notifications Why Cassandra? • Efficient writes • Durable • Scalable • High availability ACE Tuesday, March 25, 14
19. Durable + efficient writes Memory Hard drive Memtable write( , ) k1 c1:v1 Commit log Tuesday, March 25, 14
20. Memory Hard drive Memtable write( , k1 c1:v Commit log k1 c1:v k1 c1:v Tuesday, March 25, 14
21. Memory Hard drive write( , k1 c2:v k1 c1:v k1 c1:v k1 c2:v c2:v Tuesday, March 25, 14
22. Memory Hard drive k1 c1:v k1 c1:v k1 c2:v c2:v write( , ) k2 c1:v c2:v k2 c1:v c2:v k2 c1:v c2:v Tuesday, March 25, 14
23. Memory Hard drive k1 c1:v k1 c1:v k1 c2:v c2:v write( , ) k1 c1:v c3:v k2 c1:v c2:v k2 c1:v c2:v k1 c1:v c3:v c3:v Tuesday, March 25, 14
24. Memory Hard drive SSTable flush k1 c1:v c2:v k2 c1:v c2:v c3:v index / BF cleanup Tuesday, March 25, 14
25. High availability •99.9999% availability on Cassandra •(We’ll come back to this, too) Tuesday, March 25, 14
26. Core values •Massive scalability •High performance •Ease of use •Reliability/ Cassandra HBase Redis MySQL Tuesday, March 25, 14
27. 0 20000 40000 60000 80000 0 2 4 6 8 10 12 Cassandra HBase Redis MySQL NUMBER OF NODES THROUGHPUT OPS/SEC) CASSANDRA VLDB benchmark (RWS) Tuesday, March 25, 14
28. 0 8750 17500 26250 35000 1 2 4 8 16 32 Cassandra HBase MongoDB CA SSA N D RA Endpoint benchmark (RW) THROUGHPUT OPS/SEC) NUMBER OF NODES Tuesday, March 25, 14
29. Ease of use CREATE TABLE users ( id uuid PRIMARY KEY, name text, state text, birth_date int ); CREATE INDEX ON users(state); SELECT * FROM users WHERE state=‘Texas’ AND birth_date > 1950; Tuesday, March 25, 14
30. Tuesday, March 25, 14
31. Classic partitioning (SPOF) partition 1 partition 2 partition 3 partition 4 router client Tuesday, March 25, 14
32. (Not a theoretical problem) https://speakerdeck.com/mitsuhiko/a-year-of-mongodb http://aphyr.com/posts/288-the-network-is-reliable Tuesday, March 25, 14
33. Fully distributed, no SPOF p1 p1 p1 p3 p6 Client Tuesday, March 25, 14
34. Primary key determines placement* Partitioning jim carol johnny suzy age: 36 car: camaro gender: M age: 37 car: subaru gender: F age:12 gender: M age:10 gender: F Tuesday, March 25, 14
35. jim carol johnny suzy PK 5e02739678... a9a0198010... f4eb27cea7... 78b421309e... Murmur Hash Murmur* hash operation yields a 64-bit number for keys of any size. Tuesday, March 25, 14
36. Node D Node C Node B Node A The “token ring” Tuesday, March 25, 14
37. jim 5e02739678... carol a9a0198010... johnny f4eb27cea7... suzy 78b421309e... Start End A 0xc000000000..1 0x0000000000..0 B 0x0000000000..1 0x4000000000..0 C 0x4000000000..1 0x8000000000..0 D 0x8000000000..1 0xc000000000..0 Tuesday, March 25, 14
38. jim 5e02739678... carol a9a0198010... johnny f4eb27cea7... suzy 78b421309e... Start End A 0xc000000000..1 0x0000000000..0 B 0x0000000000..1 0x4000000000..0 C 0x4000000000..1 0x8000000000..0 D 0x8000000000..1 0xc000000000..0 Tuesday, March 25, 14
39. jim 5e02739678... carol a9a0198010... johnny f4eb27cea7... suzy 78b421309e... Start End A 0xc000000000..1 0x0000000000..0 B 0x0000000000..1 0x4000000000..0 C 0x4000000000..1 0x8000000000..0 D 0x8000000000..1 0xc000000000..0 Tuesday, March 25, 14
40. jim 5e02739678... carol a9a0198010... johnny f4eb27cea7... suzy 78b421309e... Start End A 0xc000000000..1 0x0000000000..0 B 0x0000000000..1 0x4000000000..0 C 0x4000000000..1 0x8000000000..0 D 0x8000000000..1 0xc000000000..0 Tuesday, March 25, 14
41. jim 5e02739678... carol a9a0198010... johnny f4eb27cea7... suzy 78b421309e... Start End A 0xc000000000..1 0x0000000000..0 B 0x0000000000..1 0x4000000000..0 C 0x4000000000..1 0x8000000000..0 D 0x8000000000..1 0xc000000000..0 Tuesday, March 25, 14
42. Node D Node C Node B Node A carol a9a0198010... Replication Tuesday, March 25, 14
43. Node D Node C Node B Node A carol a9a0198010... Tuesday, March 25, 14
44. Node D Node C Node B Node A carol a9a0198010... Tuesday, March 25, 14
45. C’’ A’’ D’ C’ A’ D A B’ C B Virtual nodes Node D Node C Node B Node A Without vnodes With vnodes Tuesday, March 25, 14
46. A closer look at reads Client Coordinator 40% busy 90% busy 30% busy Tuesday, March 25, 14
47. A closer look at reads Client Coordinator 40% busy 90% busy 30% busy Tuesday, March 25, 14
48. A closer look at reads Client Coordinator 40% busy 90% busy 30% busy Tuesday, March 25, 14
49. A closer look at reads Client Coordinator 40% busy 90% busy 30% busy Tuesday, March 25, 14
50. A closer look at reads Client Coordinator 40% busy 90% busy 30% busy Tuesday, March 25, 14
51. Consistency levels Client Coordinator 40% busy 90% busy 30% busy Tuesday, March 25, 14
52. Consistency levels Client Coordinator 40% busy 90% busy 30% busy Tuesday, March 25, 14
53. Consistency levels Client Coordinator 40% busy 90% busy 30% busy Tuesday, March 25, 14
54. Consistency levels Client Coordinator 40% busy 90% busy 30% busy Tuesday, March 25, 14
55. Consistency levels Client Coordinator 40% busy 90% busy 30% busy Tuesday, March 25, 14
56. Consistency levels •ONE •QUORUM •LOCAL_QUORUM •EACH_QUORUM •ALL Tuesday, March 25, 14
57. #CASSANDRAEU Race condition SELECT name FROM users WHERE username = 'pmcfadin'; Tuesday, March 25, 14
58. #CASSANDRAEU Race condition SELECT name FROM users WHERE username = 'pmcfadin'; (0 rows) SELECT name FROM users WHERE username = 'pmcfadin'; Tuesday, March 25, 14
59. #CASSANDRAEU Race condition SELECT name FROM users WHERE username = 'pmcfadin'; (0 rows) SELECT name FROM users WHERE username = 'pmcfadin'; INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ba27e03fd9...', '2011-06-20 13:50:00'); (0 rows) Tuesday, March 25, 14
60. #CASSANDRAEU Race condition SELECT name FROM users WHERE username = 'pmcfadin'; (0 rows) SELECT name FROM users WHERE username = 'pmcfadin'; INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ba27e03fd9...', '2011-06-20 13:50:00'); (0 rows) INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ea24e13ad9...', '2011-06-20 13:50:01'); Tuesday, March 25, 14
61. #CASSANDRAEU Race condition SELECT name FROM users WHERE username = 'pmcfadin'; This one wins (0 rows) SELECT name FROM users WHERE username = 'pmcfadin'; INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ba27e03fd9...', '2011-06-20 13:50:00'); (0 rows) INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ea24e13ad9...', '2011-06-20 13:50:01'); Tuesday, March 25, 14
62. #CASSANDRAEU Lightweight transactions INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ba27e03fd9...', '2011-06-20 13:50:00') IF NOT EXISTS; Tuesday, March 25, 14
63. #CASSANDRAEU Lightweight transactions INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ba27e03fd9...', '2011-06-20 13:50:00') IF NOT EXISTS; [applied] ----------- INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ea24e13ad9...', '2011-06-20 13:50:01') IF NOT EXISTS; Tuesday, March 25, 14
64. #CASSANDRAEU Lightweight transactions [applied] | username | created_date | name -----------+----------+----------------+---------------- False | pmcfadin | 2011-06-20 ... | Patrick McFadin INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ba27e03fd9...', '2011-06-20 13:50:00') IF NOT EXISTS; [applied] ----------- INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ea24e13ad9...', '2011-06-20 13:50:01') IF NOT EXISTS; Tuesday, March 25, 14
65. Paxos •All operations are quorum-based •Each replica sends information about unfinished operations to the leader during prepare •Paxos made Simple Tuesday, March 25, 14
66. Details •4 round trips vs 1 for normal updates •Paxos state is durable •Immediate consistency with no leader election or failover •ConsistencyLevel.SERIAL •http://www.datastax.com/dev/blog/lightweight- transactions-in-cassandra-2-0 Tuesday, March 25, 14
67. Use with caution •Great for 1% of your application •Eventual consistency is your friend •http://www.slideshare.net/planetcassandra/c- summit-2013-eventual-consistency-hopeful-consistency- by-christos-kalantzis Tuesday, March 25, 14
68. Questions? tyler@datastax.com @tylhobbs Tuesday, March 25, 14