Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scaling Your Data Safely with Riak

Alex Moore
January 10, 2014

Scaling Your Data Safely with Riak

CodeMash 2014 Presentation.

Alex Moore

January 10, 2014
Tweet

Other Decks in Programming

Transcript

  1. Internet CDN 6 8 2 3 2 1 3 3

    3? 4? q User Session Widgets Lolcats Stuff 8? magic
  2. Internet CDN 6 8 2 3 2 1 3 3

    3? 4? q User Session Widgets Lolcats Stuff 8? magic
  3. Internet CDN 6 8 2 3 2 1 3 3

    3? 4? q User Session Widgets Lolcats Stuff 8? magic
  4. Internet CDN 6 8 2 3 2 1 3 3

    3? 4? q User Session Widgets Lolcats Stuff 8? magic Who’s on first?
  5. Internet CDN 6 8 2 3 2 1 3 3

    3? 4? q User Session Widgets Lolcats Stuff 8? magic What the hell have you built? Who’s on first?
  6. 1   2   3   4   5  

    6   7   8   9   10   11   12 Building A Cluster #  All  Nodes   $  riak  start   ! #  Nodes  2..n.each     $  riak-­‐admin  join  [email protected]   ! #  Any  Node   $  riak-­‐admin  cluster  plan   $  riak-­‐admin  cluster  commit   !
  7. node 0 node 1 node 2 Replicas are stored to

    the N - 1 contiguous partitions
  8. get(“users/clay-davis”) Get Handler (FSM) client Riak hash(“users/clay-davis”) == 10, 11,

    12 Coordinating node Cluster 6 7 8 9 10 11 12 13 14 15 16 The Ring
  9. merkle tree to track changes coordinated at the vnode level

    runs as a background process exchange with neighbor vnodes for inconsistencies resolution semantics: trigger read-repair
  10. key value key value key value key value key value

    key value key value key value key value Keys are namespaced into Buckets
  11. 1   2   3   4   5  

    6   7   8   9   10   11   12 Working With Ruby - Starting $  gem  install  riak-­‐client   ! book  =  {          isbn:  '1111979723',          title:  'Moby  Dick',          author:  'Herman  Melville',          body:  'Call  me  Ishmael.  Some  years  ago...',          copies_owned:  3   }  
  12. 1   2   3   4   5  

    6   7   8   9   10   11   12 Working With Ruby - Create #  Starting  Client   client  =  Riak::Client.new(protocol:   'pbc',                           pb_port:     8087,                         host:       '10.0.0.1')   ! books_bucket  =  client.bucket('books')     ! robj_moby  =  books_bucket.new(book[:isbn])   robj_moby.data  =  book   robj_moby.store   ! !
  13. 1   2   3   4   5  

    6   7   8   9   10   11   12 Working With Ruby - Read fetched_book  =  books_bucket.get('1111979723')   ! puts  fetched_book.raw_data   !  =>  {"isbn":"1111979723","title":"Moby  Dick”,   "author":"Herman  Melville",   "body":"Call  me  Ishmael.  Some  years   ago...","copies_owned":3}   ! fetched_book.data['isbn']   !  =>  "1111979723"
  14. 1   2   3   4   5  

    6   7   8   9   10   11   12 Working With Ruby - Delete fetched_book.delete   ! #  or   ! books_bucket.delete('1111979723')  
  15. 1   2   3   4   5  

    6   7   8   9   10   11   12 Working With Ruby - 2I ! fetched_book.indexes['author_bin']  =  []   fetched_book.indexes['author_bin']  <<               fetched_book.data['author']   ! fetched_book.store   ! books_bucket.get_index('author_bin','Herman   Melville')   ! =>  ["1111979723"]
  16. 1   2   3   4   5  

    6   7   8   9   10   11   12 client.create_search_index(“tweets_idx”)   ! bucket  =  Riak::Bucket.new(client,  "tweets")   bucket.props  =  {"search_index"  =>  "tweets_idx"}   ! #  load  some  tweets  into  riak…   ! results  =  client.search("tweets",  "text:who")   ! puts  results[‘docs'].first  
  17. 1   2   3   4   5  

    6   7   8   9   10   11   12 pp  results[‘docs'].first   ! {"score"=>"2.50804900000000019489e+00",    "_yz_rb"=>"tweets",    "_yz_rk"=>"421672897250226176",    ...}