Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Cascalog
Search
αλεx π
May 22, 2013
Technology
4
130
Cascalog
Short demo talk on Cascalog on Hadoop UG in Munich
αλεx π
May 22, 2013
Tweet
Share
More Decks by αλεx π
See All by αλεx π
Scalable Time Series With Cassandra
ifesdjeen
1
350
Bayesian Inference is known to make machines biased
ifesdjeen
2
350
Cassandra for Data Analytics Backends
ifesdjeen
7
410
Stream Processing and Functional Programming
ifesdjeen
1
720
PolyConf 2015 - Rocking the Time Series boat with C, Haskell and ClojureScript
ifesdjeen
0
440
Clojure - A Sweetspot for Analytics
ifesdjeen
8
2k
Going Off Heap
ifesdjeen
3
1.9k
Always be learning
ifesdjeen
1
140
Learn Yourself Emacs For Great Good workshop slides
ifesdjeen
3
320
Other Decks in Technology
See All in Technology
Automated Promptingを目指すその前に / Before we can aim for Automated Prompting
rkaga
0
110
話題のGraphRAG、その可能性と課題を理解する
hide212131
4
1.5k
Commitment vs Harrisonism - Keynote for Scrum Niseko 2024
miholovesq
6
1.1k
Oracle Cloud Infrastructureデータベース・クラウド:各バージョンのサポート期間
oracle4engineer
PRO
27
12k
独自ツール開発でスタジオ撮影をDX!「VLS(Virtual LED Studio)」 / dx-studio-vls
cyberagentdevelopers
PRO
1
180
AWSコンテナ本出版から3年経った今、もし改めて執筆し直すなら / If I revise our container book
iselegant
15
3.9k
わたしとトラックポイント / TrackPoint tips
masahirokawahara
1
240
新R25、乃木坂46 Mobileなどのファンビジネスを支えるマルチテナンシーなプラットフォームの全体像 / cam-multi-cloud
cyberagentdevelopers
PRO
1
130
分布で見る効果検証入門 / ai-distributional-effect
cyberagentdevelopers
PRO
4
700
よくわからんサービスについての問い合わせが来たときの強い味方 Amazon Q について
kazzpapa3
0
220
「最高のチューニング」をしないために / hack@delta 24.10
fujiwara3
21
3.4k
顧客が本当に必要だったもの - パフォーマンス改善編 / Make what is needed
soudai
24
6.7k
Featured
See All Featured
Ruby is Unlike a Banana
tanoku
96
11k
The Pragmatic Product Professional
lauravandoore
31
6.3k
Building Applications with DynamoDB
mza
90
6.1k
Why Our Code Smells
bkeepers
PRO
334
57k
Docker and Python
trallard
40
3.1k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
42
9.2k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
7
150
How to train your dragon (web standard)
notwaldorf
88
5.7k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
364
22k
What's new in Ruby 2.0
geeforr
342
31k
Build your cross-platform service in a week with App Engine
jlugia
229
18k
Art, The Web, and Tiny UX
lynnandtonic
296
20k
Transcript
Cascalog Hassle-free MapReduce that matches your scale Thursday, May 23,
13
Thursday, May 23, 13
Setting expecations •This is not a guide •And not a
tutorial •Doesn’t claim to be complete •Mostly to give you an idea •And encourage you to explore further Thursday, May 23, 13
How much time do you spend on writing logic that
framework should take care of? Thursday, May 23, 13
How easy is it to debug your map/reduce aggragation? Thursday,
May 23, 13
Hadoop + Java composable, but too vebrose Pig, Hive too
concrete, lack of abstraction and composition Thursday, May 23, 13
Thursday, May 23, 13
• Clear, declarative syntax • Inner and outer joins •
Aggregators • Functions • Subqueries, composition • Sorting • Performant Thursday, May 23, 13
Casca-WHAT? • Built on top of Hadoop (MapReduce) • Cascading
(tuples, workflows, job execution) • Written in Clojure • Datalog (logic programming) Thursday, May 23, 13
Abstract evrthn! Thursday, May 23, 13
Source where data pours from Thursday, May 23, 13
Pipe that data flows through Thursday, May 23, 13
Filter that makes sure that only good stuff goes through
Thursday, May 23, 13
Tuple they actually flow Thursday, May 23, 13
Thursday, May 23, 13
Query anatomy Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Output Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
output vars Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Input input vars Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Logic/aggregations Thursday, May 23, 13
Sources and sinks • HDFS (go figure) • Cassandra •
MongoDB • SQL data sources • File system • Memory sources Thursday, May 23, 13
(?<- (stdout) [?person] (age ?person 25)) Exact match of second
element in a tuple Thursday, May 23, 13
(defn younger-than? [limit age] (< age limit)) (?<- (stdout) [?person
?age] (age ?person ?age) (younger-than? 32 ?age)) Predicate match, fn call Predicate Thursday, May 23, 13
(?<- (stdout) [?person ?count] (follows ?person _) (c/count ?count)) Aggregation
Thursday, May 23, 13
SHOWTIME! Thursday, May 23, 13
Benefits •Query language is same as application language •Subqueries, reusability
•Ad-hoc querying •Cascading underneath, so taps for all DBs work •Reuse application logic •Text editor integration Thursday, May 23, 13
@ifesdjeen (twitter/github) Thursday, May 23, 13