• Image and audio compression trade CPU/ fidelity loss for space • Databases trade consistency for availability in failure scenarios Tuesday, June 26, 12
partition (message loss) occurs, laws of physics make you choose: • Consistency OR • Availability • No system can “beat the CAP theorem” Tuesday, June 26, 12
of the key • Provides even distribution of storage and query load • Trades off advantages gained from locality • range queries • aggregates Tuesday, June 26, 12
Each host manages many vnodes • Riak *could* manage all host-local storage as a unit and gain efficiency, but would lose • simplicity in cluster resizing • failure isolation Tuesday, June 26, 12
• This provides crash-safety, fast writes • Tradeoff: must periodically compact/merge files to reclaim space • Causes periodic pauses while compaction occurs that must be masked/mitigated Tuesday, June 26, 12
”keydir” is updated. A keydir is simply a hash table that maps every key in a Bitcask to a fixed-size structure giving the file, offset, and size of the most recently written entry for that key. When a write occurs, the keydir is atomically updated with the location of the newest data. The old data is still present on disk, but any new reads will use the latest version available in the keydir. As we’ll see later, the merge process will eventually remove the old value. Reading a value is simple, and doesn’t ever require more than a single disk seek. We look up the key in our keydir, and from there we read the data using the file id, position, and size that are returned from that lookup. In many cases, the operating system’s filesystem read-ahead cache makes this a much faster operation than would be otherwise expected. Tradeoff: Index must fit in memory Low Latency: All reads = hash lookup + 1 seek All writes = append to file Tuesday, June 26, 12
cluster, data must be rebalanced • Rebalancing causes disk, network load • Tradeoff: speed of convergence vs. effects on cluster performance Tuesday, June 26, 12
N=3, R=W=2 • Bad for microbenchmarks, good for production use, durability • Mongo ships with W=0 • Good for benchmarks, horrible and insane for durability, production use. Tuesday, June 26, 12