Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
A Riak Query Tale
Search
Mathias Meyer
February 01, 2012
Programming
5
1k
A Riak Query Tale
An introduction to the abundance of ways you can get data out of Riak.
Mathias Meyer
February 01, 2012
Tweet
Share
More Decks by Mathias Meyer
See All by Mathias Meyer
Building and Scaling an Distributed and Inclusive Team
roidrage
0
1.3k
cooking infrastructure with chef
roidrage
4
230
The Message Queue is Dead, Long Live the Message Queue
roidrage
4
700
riak-js
roidrage
1
280
designing for concurrency with riak
roidrage
11
1.8k
metrics, monitoring, logging
roidrage
82
15k
design for cloud - jax 2012
roidrage
2
300
Don't Use NoSQL
roidrage
10
1.1k
Designing Applications for Amazon Web Services (GOTO Aarhus)
roidrage
6
360
Other Decks in Programming
See All in Programming
Understanding Kotlin Multiplatform
l2hyunwoo
0
250
#QiitaBash TDDで(自分の)開発がどう変わったか
ryosukedtomita
1
350
Amazon Q CLI開発で学んだAIコーディングツールの使い方
licux
3
170
QA x AIエコシステム段階構築作戦
osu
0
240
実践 Dev Containers × Claude Code
touyu
1
120
なぜあなたのオブザーバビリティ導入は頓挫するのか
ryota_hnk
5
560
あまり知られていない MCP 仕様たち / MCP specifications that aren’t widely known
ktr_0731
0
220
オホーツクでコミュニティを立ち上げた理由―地方出身プログラマの挑戦 / TechRAMEN 2025 Conference
lemonade_37
1
430
No Install CMS戦略 〜 5年先を見据えたフロントエンド開発を考える / no_install_cms
rdlabo
0
430
Android 15以上でPDFのテキスト検索を爆速開発!
tonionagauzzi
0
180
バイブコーディング超えてバイブデプロイ〜CloudflareMCPで実現する、未来のアプリケーションデリバリー〜
azukiazusa1
3
780
副作用と戦う PHP リファクタリング ─ ドメインイベントでビジネスロジックを解きほぐす
kajitack
3
520
Featured
See All Featured
The Invisible Side of Design
smashingmag
301
51k
A better future with KSS
kneath
238
17k
Producing Creativity
orderedlist
PRO
346
40k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
161
15k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.6k
Navigating Team Friction
lara
188
15k
The Cost Of JavaScript in 2023
addyosmani
51
8.7k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
139
34k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
3k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.4k
Transcript
A Riak Query Tale Mathias Meyer, @roidrage NoSQL Cologne
http://riakhandbook.com
Riak Distributed Database Fault-Tolerant Content-Agnostic Scalable on Demand
Querying Data
Key-Value $ curl localhost:8098/riak/users/roidrage
Links $ curl -‐v localhost:8098/riak/users/roidrage < HTTP/1.1 200 OK <
Link: </riak/users/klimpong>; riaktag="friend"
Links $ curl .../riak/users/roidrage/users,friend,_/
Listing Keys $ curl .../riak/users?keys=true
Don’t do that!
Streaming Keys $ curl .../riak/users?keys=stream
Avoid that!
Loads all the keys.
MapReduce
MapReduce Transform (Map) Aggregate (Reduce)
Warning: JavaScript
MapReduce riak.add("users"). map("Riak.mapValues").
run()
MapReduce var nameLength = function(value) { var doc
= Riak.mapValues(value)[0]; return [doc.length]; }
MapReduce riak.add("users"). map(nameLength).
run()
MapReduce riak.add("users"). map(nameLength).
reduce("Riak.reduceSum"). run()
MapReduce var average = function(values) { var avg
= values.reduce(function(n, sum) { return sum += n; }, 0); return [(avg / values.length)]; }
MapReduce riak.add("users"). map(nameLength).
reduce(average). run()
MapReduce riak.add("users"). map(nameLength).
reduce(average). run() Uh-Oh!
MapReduce riak.add(["users", "roidrage"]). map(nameLength).
reduce(average). run() Better!
JavaScript M/R Breaks with Millions of Objects Uses External Libraries
Serializes Data for JavaScript
Warning: Erlang
MapReduce riak.add('tweets'). map({language: 'erlang',
module: 'riak_kv_mapreduce', function: 'map_object_value'}).run()
MapReduce $ riak attach > {ok, C} = riak:local_client().
MapReduce C:mapred([{<<"users">>, <<"roidrage">>}], [{map, {modfun, riak_kv_mapreduce, map_object_value}, none, false}, {reduce,
{modfun, riak_kv_mapreduce, reduce_count_inputs}, none, true}]).
MapReduce ExtractFirstName1 = fun(RObject, _, _) -‐>
Value = riak_object:get_value(RObject), [FirstName, _] = re:split(Value, " "), [FirstName] end.
MapReduce C:mapred([{<<"users">>, <<"roidrage">>}],
[{map, {qfun, ExtractFirstName}, none, true}]).
Erlang M/R Much more efficient than JavaScript No serialization No
ad-hoc functions through HTTP
Key-Filters Reduce MapReduce input Based on key matches
Key-Filters riak.add({bucket: 'users', key_filters: [["matches", "^roid"]]})
Key-Filters riak.add({bucket: 'users', key_filters: [["to_upper"],
["matches", "^ROID"]]})
Key-Filters riak.add({bucket: 'users', key_filters: [["to_upper"],
["to_lower"], ["matches", "^roid"]]})
Key-Filters riak.add({bucket: 'users', key_filters: [["to_upper"],
["ends_with", "RAGE"]]})
Key-Filters riak.add({bucket: 'users', key_filters:
[["and", [["string_to_int"], ["less_than", 10]], [["string_to_int"], ["greater_than", 5]]]]})
Don't use key filters.
Riak 2i Sorted Secondary Indexes Simple Reverse Lookups Maintained Manually
Requires LevelDB
Riak 2i curl -‐X PUT .../riak/users/roidrage -‐d @-‐ \
-‐H "Content-‐Type: text/plain" \ -‐H "X-‐Riak-‐Index-‐firstname_bin: mathias" \ -‐H "X-‐Riak-‐Index-‐lastname_bin: meyer"
Riak 2i X-‐Riak-‐Index-‐firstname_bin: Mathias X-‐Riak-‐Index-‐lastname_bin: Meyer
Riak 2i X-‐Riak-‐Index-‐firstname_bin: Mathias X-‐Riak-‐Index-‐lastname_bin: Meyer X-‐Riak-‐Index-‐age_int: 34
Riak 2i X-‐Riak-‐Index-‐firstname_bin: Mathias X-‐Riak-‐Index-‐lastname_bin: Meyer X-‐Riak-‐Index-‐age_int: 34 X-‐Riak-‐Index-‐topics_bin: nosql,cloud,operations
Riak 2i # Match $ curl .../buckets/users/index/firstname_bin/Mathias
Riak 2i # Range $ curl .../buckets/users/index/firstname_bin/Mathias/Till
Riak 2i # Key $ curl .../buckets/users/index/$key/roidrage
Ordered Keys! (sort of)
MapReduce riak.add({bucket: 'users',
index: 'lastname_bin', key: 'mathias'}). map('Riak.mapValuesJson').run()
Riak 2i No Multi-Index Queries Requires Extra Work in the
App Returns only keys Document-partitioned
Riak Search Full-Text Search Solr-ish Interface Integrates with Riak
Riak Search curl -‐X PUT localhost:8098/riak/users -‐d @-‐ \
-‐H "Content-‐Type: application/json" {"props":{"precommit": [{"mod":"riak_search_kv_hook","fun":"precommit"} ]}}
Indexing Riak Objects curl -‐X PUT .../riak/users/roidrage \
-‐d "Mathias Meyer" -‐H "Content-‐Type: text/plain"
Solr-ish Interface curl .../solr/users/select?q=value:Mathias
Riak Search value:Mathias OR value:Till value:Mathias AND value:Meyer value:Mat* value:[Mathias
TO Till]
MapReduce riak.addSearch("users", "value:Mathias"). map("Riak.mapValues").run()
Riak Search Full text search of structured data Term-partitioned Efficient
for one term queries Multiple Interfaces No Anti-Entropy
When?
Key Listings Never! Almost
MapReduce Analytical Queries Fixed Dataset
Key Filters Never!
Riak 2i Simple Lookups and Range Queries Unbounded Queries Full
Fault-Tolerance
Riak Search Larger documents Full indexing Flexible queries Low frequency
terms
Questions?