Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Cascalog
Search
αλεx π
May 22, 2013
Technology
4
140
Cascalog
Short demo talk on Cascalog on Hadoop UG in Munich
αλεx π
May 22, 2013
Tweet
Share
More Decks by αλεx π
See All by αλεx π
Scalable Time Series With Cassandra
ifesdjeen
1
360
Bayesian Inference is known to make machines biased
ifesdjeen
2
360
Cassandra for Data Analytics Backends
ifesdjeen
7
420
Stream Processing and Functional Programming
ifesdjeen
1
730
PolyConf 2015 - Rocking the Time Series boat with C, Haskell and ClojureScript
ifesdjeen
0
450
Clojure - A Sweetspot for Analytics
ifesdjeen
8
2k
Going Off Heap
ifesdjeen
3
1.9k
Always be learning
ifesdjeen
1
140
Learn Yourself Emacs For Great Good workshop slides
ifesdjeen
3
330
Other Decks in Technology
See All in Technology
MCP で繋ぐ Figma とデザインシステム〜LLM を使った UI 実装のリアル〜
kimuson
2
1.3k
CSS polyfill とその未来
ken7253
0
140
エンジニアが組織に馴染むために勉強会を主催してチームの壁を越える
ohmori_yusuke
2
120
Eight Engineering Unit 紹介資料
sansan33
PRO
0
3.2k
ソフトウェアテストのAI活用_ver1.10
fumisuke
0
240
アプリケーションの中身が見える!Mackerel APMの全貌と展望 / Mackerel APMリリースパーティ
mackerelio
0
450
実践Kafka Streams 〜イベント駆動型アーキテクチャを添えて〜
joker1007
0
460
Java で学ぶ 代数的データ型
ysknsid25
1
530
大手企業のAIツール導入の壁を越えて:サイバーエージェントのCursor活用戦略
gunta
19
5.4k
人とAIとの共創を夢見た2か月 #共創AIミートアップ / Co-Creation with Keito-chan
kondoyuko
1
710
Introduction to Bill One Development Engineer
sansan33
PRO
0
240
Javaアプリケーションの配布とパッケージング / Distribution and packaging of Java applications
hogelog
1
250
Featured
See All Featured
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
It's Worth the Effort
3n
184
28k
Build The Right Thing And Hit Your Dates
maggiecrowley
35
2.7k
[RailsConf 2023] Rails as a piece of cake
palkan
55
5.6k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
12k
Statistics for Hackers
jakevdp
799
220k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.7k
VelocityConf: Rendering Performance Case Studies
addyosmani
329
24k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
6
660
A designer walks into a library…
pauljervisheath
205
24k
Designing Experiences People Love
moore
142
24k
Why Our Code Smells
bkeepers
PRO
336
57k
Transcript
Cascalog Hassle-free MapReduce that matches your scale Thursday, May 23,
13
Thursday, May 23, 13
Setting expecations •This is not a guide •And not a
tutorial •Doesn’t claim to be complete •Mostly to give you an idea •And encourage you to explore further Thursday, May 23, 13
How much time do you spend on writing logic that
framework should take care of? Thursday, May 23, 13
How easy is it to debug your map/reduce aggragation? Thursday,
May 23, 13
Hadoop + Java composable, but too vebrose Pig, Hive too
concrete, lack of abstraction and composition Thursday, May 23, 13
Thursday, May 23, 13
• Clear, declarative syntax • Inner and outer joins •
Aggregators • Functions • Subqueries, composition • Sorting • Performant Thursday, May 23, 13
Casca-WHAT? • Built on top of Hadoop (MapReduce) • Cascading
(tuples, workflows, job execution) • Written in Clojure • Datalog (logic programming) Thursday, May 23, 13
Abstract evrthn! Thursday, May 23, 13
Source where data pours from Thursday, May 23, 13
Pipe that data flows through Thursday, May 23, 13
Filter that makes sure that only good stuff goes through
Thursday, May 23, 13
Tuple they actually flow Thursday, May 23, 13
Thursday, May 23, 13
Query anatomy Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Output Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
output vars Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Input input vars Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Logic/aggregations Thursday, May 23, 13
Sources and sinks • HDFS (go figure) • Cassandra •
MongoDB • SQL data sources • File system • Memory sources Thursday, May 23, 13
(?<- (stdout) [?person] (age ?person 25)) Exact match of second
element in a tuple Thursday, May 23, 13
(defn younger-than? [limit age] (< age limit)) (?<- (stdout) [?person
?age] (age ?person ?age) (younger-than? 32 ?age)) Predicate match, fn call Predicate Thursday, May 23, 13
(?<- (stdout) [?person ?count] (follows ?person _) (c/count ?count)) Aggregation
Thursday, May 23, 13
SHOWTIME! Thursday, May 23, 13
Benefits •Query language is same as application language •Subqueries, reusability
•Ad-hoc querying •Cascading underneath, so taps for all DBs work •Reuse application logic •Text editor integration Thursday, May 23, 13
@ifesdjeen (twitter/github) Thursday, May 23, 13