Share
Embed
Intro to Cassandra
Presented at a joint meeting of the Austin Java Users Group and the Austin Python Users Group on March 25, 2014.
Tyler Hobbs
March 25, 2014
Transcript
68 slides
-
©2014 DataStax Cassandra committer at DataStax [email protected] @tylhobbs Tyler Hobbs
Intro to Cassandra 1 Tuesday, March 25, 14 -
Five years of Cassandra Jul-09 May-10 Feb-11 Dec-11 Oct-12 Jul-13
0.1 0.3 0.6 0.7 1.0 1.2 ... 2.0 DSE Jul-08 Tuesday, March 25, 14 -
Application/Use Case • Social Signals: like/want/own features for eBay product
and item pages • Hunch taste graph for eBay users and items • Many time series use cases Why Cassandra? • Multi-datacenter • Scalable • Write performance • Distributed counters • Hadoop support ACE Tuesday, March 25, 14 -
Application/Use Case • Adobe AudienceManager: web analytics, content management, and
online advertising Why Cassandra? • Low-latency • Scalable • Multi-datacenter • Tuneable consistency ACE Tuesday, March 25, 14 -
Bootstrapping s d s d s d s d Tuesday,
March 25, 14 -
Bootstrapping s d s d s d s d Tuesday,
March 25, 14 -
Application/Use Case • Logging • Notifications Why Cassandra? • Efficient
writes • Durable • Scalable • High availability ACE Tuesday, March 25, 14 -
Durable + efficient writes Memory Hard drive Memtable write( ,
) k1 c1:v1 Commit log Tuesday, March 25, 14 -
Memory Hard drive Memtable write( , k1 c1:v Commit log
k1 c1:v k1 c1:v Tuesday, March 25, 14 -
Memory Hard drive write( , k1 c2:v k1 c1:v k1
c1:v k1 c2:v c2:v Tuesday, March 25, 14 -
Memory Hard drive k1 c1:v k1 c1:v k1 c2:v c2:v
write( , ) k2 c1:v c2:v k2 c1:v c2:v k2 c1:v c2:v Tuesday, March 25, 14 -
Memory Hard drive k1 c1:v k1 c1:v k1 c2:v c2:v
write( , ) k1 c1:v c3:v k2 c1:v c2:v k2 c1:v c2:v k1 c1:v c3:v c3:v Tuesday, March 25, 14 -
Memory Hard drive SSTable flush k1 c1:v c2:v k2 c1:v
c2:v c3:v index / BF cleanup Tuesday, March 25, 14 -
High availability •99.9999% availability on Cassandra •(We’ll come back to
this, too) Tuesday, March 25, 14 -
Core values •Massive scalability •High performance •Ease of use •Reliability/
Cassandra HBase Redis MySQL Tuesday, March 25, 14 -
0 20000 40000 60000 80000 0 2 4 6 8
10 12 Cassandra HBase Redis MySQL NUMBER OF NODES THROUGHPUT OPS/SEC) CASSANDRA VLDB benchmark (RWS) Tuesday, March 25, 14 -
0 8750 17500 26250 35000 1 2 4 8 16
32 Cassandra HBase MongoDB CA SSA N D RA Endpoint benchmark (RW) THROUGHPUT OPS/SEC) NUMBER OF NODES Tuesday, March 25, 14 -
Ease of use CREATE TABLE users ( id uuid PRIMARY
KEY, name text, state text, birth_date int ); CREATE INDEX ON users(state); SELECT * FROM users WHERE state=‘Texas’ AND birth_date > 1950; Tuesday, March 25, 14 -
Classic partitioning (SPOF) partition 1 partition 2 partition 3 partition
4 router client Tuesday, March 25, 14 -
Fully distributed, no SPOF p1 p1 p1 p3 p6 Client
Tuesday, March 25, 14 -
Primary key determines placement* Partitioning jim carol johnny suzy age:
36 car: camaro gender: M age: 37 car: subaru gender: F age:12 gender: M age:10 gender: F Tuesday, March 25, 14 -
jim carol johnny suzy PK 5e02739678... a9a0198010... f4eb27cea7... 78b421309e... Murmur
Hash Murmur* hash operation yields a 64-bit number for keys of any size. Tuesday, March 25, 14 -
Node D Node C Node B Node A The “token
ring” Tuesday, March 25, 14 -
jim 5e02739678... carol a9a0198010... johnny f4eb27cea7... suzy 78b421309e... Start End
A 0xc000000000..1 0x0000000000..0 B 0x0000000000..1 0x4000000000..0 C 0x4000000000..1 0x8000000000..0 D 0x8000000000..1 0xc000000000..0 Tuesday, March 25, 14 -
jim 5e02739678... carol a9a0198010... johnny f4eb27cea7... suzy 78b421309e... Start End
A 0xc000000000..1 0x0000000000..0 B 0x0000000000..1 0x4000000000..0 C 0x4000000000..1 0x8000000000..0 D 0x8000000000..1 0xc000000000..0 Tuesday, March 25, 14 -
jim 5e02739678... carol a9a0198010... johnny f4eb27cea7... suzy 78b421309e... Start End
A 0xc000000000..1 0x0000000000..0 B 0x0000000000..1 0x4000000000..0 C 0x4000000000..1 0x8000000000..0 D 0x8000000000..1 0xc000000000..0 Tuesday, March 25, 14 -
jim 5e02739678... carol a9a0198010... johnny f4eb27cea7... suzy 78b421309e... Start End
A 0xc000000000..1 0x0000000000..0 B 0x0000000000..1 0x4000000000..0 C 0x4000000000..1 0x8000000000..0 D 0x8000000000..1 0xc000000000..0 Tuesday, March 25, 14 -
jim 5e02739678... carol a9a0198010... johnny f4eb27cea7... suzy 78b421309e... Start End
A 0xc000000000..1 0x0000000000..0 B 0x0000000000..1 0x4000000000..0 C 0x4000000000..1 0x8000000000..0 D 0x8000000000..1 0xc000000000..0 Tuesday, March 25, 14 -
Node D Node C Node B Node A carol a9a0198010...
Replication Tuesday, March 25, 14 -
Node D Node C Node B Node A carol a9a0198010...
Tuesday, March 25, 14 -
Node D Node C Node B Node A carol a9a0198010...
Tuesday, March 25, 14 -
C’’ A’’ D’ C’ A’ D A B’ C B
Virtual nodes Node D Node C Node B Node A Without vnodes With vnodes Tuesday, March 25, 14 -
A closer look at reads Client Coordinator 40% busy 90%
busy 30% busy Tuesday, March 25, 14 -
A closer look at reads Client Coordinator 40% busy 90%
busy 30% busy Tuesday, March 25, 14 -
A closer look at reads Client Coordinator 40% busy 90%
busy 30% busy Tuesday, March 25, 14 -
A closer look at reads Client Coordinator 40% busy 90%
busy 30% busy Tuesday, March 25, 14 -
A closer look at reads Client Coordinator 40% busy 90%
busy 30% busy Tuesday, March 25, 14 -
Consistency levels Client Coordinator 40% busy 90% busy 30% busy
Tuesday, March 25, 14 -
Consistency levels Client Coordinator 40% busy 90% busy 30% busy
Tuesday, March 25, 14 -
Consistency levels Client Coordinator 40% busy 90% busy 30% busy
Tuesday, March 25, 14 -
Consistency levels Client Coordinator 40% busy 90% busy 30% busy
Tuesday, March 25, 14 -
Consistency levels Client Coordinator 40% busy 90% busy 30% busy
Tuesday, March 25, 14 -
#CASSANDRAEU Race condition SELECT name FROM users WHERE username =
'pmcfadin'; Tuesday, March 25, 14 -
#CASSANDRAEU Race condition SELECT name FROM users WHERE username =
'pmcfadin'; (0 rows) SELECT name FROM users WHERE username = 'pmcfadin'; Tuesday, March 25, 14 -
#CASSANDRAEU Race condition SELECT name FROM users WHERE username =
'pmcfadin'; (0 rows) SELECT name FROM users WHERE username = 'pmcfadin'; INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ba27e03fd9...', '2011-06-20 13:50:00'); (0 rows) Tuesday, March 25, 14 -
#CASSANDRAEU Race condition SELECT name FROM users WHERE username =
'pmcfadin'; (0 rows) SELECT name FROM users WHERE username = 'pmcfadin'; INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ba27e03fd9...', '2011-06-20 13:50:00'); (0 rows) INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ea24e13ad9...', '2011-06-20 13:50:01'); Tuesday, March 25, 14 -
#CASSANDRAEU Race condition SELECT name FROM users WHERE username =
'pmcfadin'; This one wins (0 rows) SELECT name FROM users WHERE username = 'pmcfadin'; INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ba27e03fd9...', '2011-06-20 13:50:00'); (0 rows) INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ea24e13ad9...', '2011-06-20 13:50:01'); Tuesday, March 25, 14 -
#CASSANDRAEU Lightweight transactions INSERT INTO users (username, name, password, created_date)
VALUES ('pmcfadin', 'Patrick McFadin', 'ba27e03fd9...', '2011-06-20 13:50:00') IF NOT EXISTS; Tuesday, March 25, 14 -
#CASSANDRAEU Lightweight transactions INSERT INTO users (username, name, password, created_date)
VALUES ('pmcfadin', 'Patrick McFadin', 'ba27e03fd9...', '2011-06-20 13:50:00') IF NOT EXISTS; [applied] ----------- INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ea24e13ad9...', '2011-06-20 13:50:01') IF NOT EXISTS; Tuesday, March 25, 14 -
#CASSANDRAEU Lightweight transactions [applied] | username | created_date | name
-----------+----------+----------------+---------------- False | pmcfadin | 2011-06-20 ... | Patrick McFadin INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ba27e03fd9...', '2011-06-20 13:50:00') IF NOT EXISTS; [applied] ----------- INSERT INTO users (username, name, password, created_date) VALUES ('pmcfadin', 'Patrick McFadin', 'ea24e13ad9...', '2011-06-20 13:50:01') IF NOT EXISTS; Tuesday, March 25, 14 -
Paxos •All operations are quorum-based •Each replica sends information about
unfinished operations to the leader during prepare •Paxos made Simple Tuesday, March 25, 14 -
Details •4 round trips vs 1 for normal updates •Paxos
state is durable •Immediate consistency with no leader election or failover •ConsistencyLevel.SERIAL •http://www.datastax.com/dev/blog/lightweight- transactions-in-cassandra-2-0 Tuesday, March 25, 14 -
Use with caution •Great for 1% of your application •Eventual
consistency is your friend •http://www.slideshare.net/planetcassandra/c- summit-2013-eventual-consistency-hopeful-consistency- by-christos-kalantzis Tuesday, March 25, 14