Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
A Riak Query Tale
Search
Mathias Meyer
February 01, 2012
Programming
5
1k
A Riak Query Tale
An introduction to the abundance of ways you can get data out of Riak.
Mathias Meyer
February 01, 2012
Tweet
Share
More Decks by Mathias Meyer
See All by Mathias Meyer
Building and Scaling an Distributed and Inclusive Team
roidrage
0
1.2k
cooking infrastructure with chef
roidrage
4
220
The Message Queue is Dead, Long Live the Message Queue
roidrage
4
680
riak-js
roidrage
1
270
designing for concurrency with riak
roidrage
11
1.8k
metrics, monitoring, logging
roidrage
82
15k
design for cloud - jax 2012
roidrage
2
290
Don't Use NoSQL
roidrage
10
1k
Designing Applications for Amazon Web Services (GOTO Aarhus)
roidrage
6
350
Other Decks in Programming
See All in Programming
TypeScriptでライブラリとの依存を限定的にする方法
tutinoko
2
660
Generative AI Use Cases JP (略称:GenU)奮闘記
hideg
1
290
よくできたテンプレート言語として TypeScript + JSX を利用する試み / Using TypeScript + JSX outside of Web Frontend #TSKaigiKansai
izumin5210
6
1.7k
CSC509 Lecture 11
javiergs
PRO
0
180
Snowflake x dbtで作るセキュアでアジャイルなデータ基盤
tsoshiro
2
520
Why Jakarta EE Matters to Spring - and Vice Versa
ivargrimstad
0
1.1k
Streams APIとTCPフロー制御 / Web Streams API and TCP flow control
tasshi
2
350
詳細解説! ArrayListの仕組みと実装
yujisoftware
0
580
Hotwire or React? ~アフタートーク・本編に含めなかった話~ / Hotwire or React? after talk
harunatsujita
1
120
Duckdb-Wasmでローカルダッシュボードを作ってみた
nkforwork
0
120
LLM生成文章の精度評価自動化とプロンプトチューニングの効率化について
layerx
PRO
2
190
色々なIaCツールを実際に触って比較してみる
iriikeita
0
330
Featured
See All Featured
Mobile First: as difficult as doing things right
swwweet
222
8.9k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
27
840
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
28
9.1k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
28
8.2k
Building Flexible Design Systems
yeseniaperezcruz
327
38k
Happy Clients
brianwarren
98
6.7k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
169
50k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
8
860
Teambox: Starting and Learning
jrom
133
8.8k
Music & Morning Musume
bryan
46
6.2k
Building Applications with DynamoDB
mza
90
6.1k
Become a Pro
speakerdeck
PRO
25
5k
Transcript
A Riak Query Tale Mathias Meyer, @roidrage NoSQL Cologne
http://riakhandbook.com
Riak Distributed Database Fault-Tolerant Content-Agnostic Scalable on Demand
Querying Data
Key-Value $ curl localhost:8098/riak/users/roidrage
Links $ curl -‐v localhost:8098/riak/users/roidrage < HTTP/1.1 200 OK <
Link: </riak/users/klimpong>; riaktag="friend"
Links $ curl .../riak/users/roidrage/users,friend,_/
Listing Keys $ curl .../riak/users?keys=true
Don’t do that!
Streaming Keys $ curl .../riak/users?keys=stream
Avoid that!
Loads all the keys.
MapReduce
MapReduce Transform (Map) Aggregate (Reduce)
Warning: JavaScript
MapReduce riak.add("users"). map("Riak.mapValues").
run()
MapReduce var nameLength = function(value) { var doc
= Riak.mapValues(value)[0]; return [doc.length]; }
MapReduce riak.add("users"). map(nameLength).
run()
MapReduce riak.add("users"). map(nameLength).
reduce("Riak.reduceSum"). run()
MapReduce var average = function(values) { var avg
= values.reduce(function(n, sum) { return sum += n; }, 0); return [(avg / values.length)]; }
MapReduce riak.add("users"). map(nameLength).
reduce(average). run()
MapReduce riak.add("users"). map(nameLength).
reduce(average). run() Uh-Oh!
MapReduce riak.add(["users", "roidrage"]). map(nameLength).
reduce(average). run() Better!
JavaScript M/R Breaks with Millions of Objects Uses External Libraries
Serializes Data for JavaScript
Warning: Erlang
MapReduce riak.add('tweets'). map({language: 'erlang',
module: 'riak_kv_mapreduce', function: 'map_object_value'}).run()
MapReduce $ riak attach > {ok, C} = riak:local_client().
MapReduce C:mapred([{<<"users">>, <<"roidrage">>}], [{map, {modfun, riak_kv_mapreduce, map_object_value}, none, false}, {reduce,
{modfun, riak_kv_mapreduce, reduce_count_inputs}, none, true}]).
MapReduce ExtractFirstName1 = fun(RObject, _, _) -‐>
Value = riak_object:get_value(RObject), [FirstName, _] = re:split(Value, " "), [FirstName] end.
MapReduce C:mapred([{<<"users">>, <<"roidrage">>}],
[{map, {qfun, ExtractFirstName}, none, true}]).
Erlang M/R Much more efficient than JavaScript No serialization No
ad-hoc functions through HTTP
Key-Filters Reduce MapReduce input Based on key matches
Key-Filters riak.add({bucket: 'users', key_filters: [["matches", "^roid"]]})
Key-Filters riak.add({bucket: 'users', key_filters: [["to_upper"],
["matches", "^ROID"]]})
Key-Filters riak.add({bucket: 'users', key_filters: [["to_upper"],
["to_lower"], ["matches", "^roid"]]})
Key-Filters riak.add({bucket: 'users', key_filters: [["to_upper"],
["ends_with", "RAGE"]]})
Key-Filters riak.add({bucket: 'users', key_filters:
[["and", [["string_to_int"], ["less_than", 10]], [["string_to_int"], ["greater_than", 5]]]]})
Don't use key filters.
Riak 2i Sorted Secondary Indexes Simple Reverse Lookups Maintained Manually
Requires LevelDB
Riak 2i curl -‐X PUT .../riak/users/roidrage -‐d @-‐ \
-‐H "Content-‐Type: text/plain" \ -‐H "X-‐Riak-‐Index-‐firstname_bin: mathias" \ -‐H "X-‐Riak-‐Index-‐lastname_bin: meyer"
Riak 2i X-‐Riak-‐Index-‐firstname_bin: Mathias X-‐Riak-‐Index-‐lastname_bin: Meyer
Riak 2i X-‐Riak-‐Index-‐firstname_bin: Mathias X-‐Riak-‐Index-‐lastname_bin: Meyer X-‐Riak-‐Index-‐age_int: 34
Riak 2i X-‐Riak-‐Index-‐firstname_bin: Mathias X-‐Riak-‐Index-‐lastname_bin: Meyer X-‐Riak-‐Index-‐age_int: 34 X-‐Riak-‐Index-‐topics_bin: nosql,cloud,operations
Riak 2i # Match $ curl .../buckets/users/index/firstname_bin/Mathias
Riak 2i # Range $ curl .../buckets/users/index/firstname_bin/Mathias/Till
Riak 2i # Key $ curl .../buckets/users/index/$key/roidrage
Ordered Keys! (sort of)
MapReduce riak.add({bucket: 'users',
index: 'lastname_bin', key: 'mathias'}). map('Riak.mapValuesJson').run()
Riak 2i No Multi-Index Queries Requires Extra Work in the
App Returns only keys Document-partitioned
Riak Search Full-Text Search Solr-ish Interface Integrates with Riak
Riak Search curl -‐X PUT localhost:8098/riak/users -‐d @-‐ \
-‐H "Content-‐Type: application/json" {"props":{"precommit": [{"mod":"riak_search_kv_hook","fun":"precommit"} ]}}
Indexing Riak Objects curl -‐X PUT .../riak/users/roidrage \
-‐d "Mathias Meyer" -‐H "Content-‐Type: text/plain"
Solr-ish Interface curl .../solr/users/select?q=value:Mathias
Riak Search value:Mathias OR value:Till value:Mathias AND value:Meyer value:Mat* value:[Mathias
TO Till]
MapReduce riak.addSearch("users", "value:Mathias"). map("Riak.mapValues").run()
Riak Search Full text search of structured data Term-partitioned Efficient
for one term queries Multiple Interfaces No Anti-Entropy
When?
Key Listings Never! Almost
MapReduce Analytical Queries Fixed Dataset
Key Filters Never!
Riak 2i Simple Lookups and Range Queries Unbounded Queries Full
Fault-Tolerance
Riak Search Larger documents Full indexing Flexible queries Low frequency
terms
Questions?