Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cascalog

Avatar for αλεx π αλεx π
May 22, 2013
Technology
4
140

 Cascalog

Short demo talk on Cascalog on Hadoop UG in Munich

Avatar for αλεx π

αλεx π

May 22, 2013
Tweet

More Decks by αλεx π

See All by αλεx π
Scalable Time Series With Cassandra
Avatar for αλεx π ifesdjeen
1
370
Bayesian Inference is known to make machines biased
Avatar for αλεx π ifesdjeen
2
370
Cassandra for Data Analytics Backends
Avatar for αλεx π ifesdjeen
7
430
Stream Processing and Functional Programming
Avatar for αλεx π ifesdjeen
1
750
PolyConf 2015 - Rocking the Time Series boat with C, Haskell and ClojureScript
Avatar for αλεx π ifesdjeen
0
470
Clojure - A Sweetspot for Analytics
Avatar for αλεx π ifesdjeen
8
2.1k
Going Off Heap
Avatar for αλεx π ifesdjeen
3
1.9k
Always be learning
Avatar for αλεx π ifesdjeen
1
140
Learn Yourself Emacs For Great Good workshop slides
Avatar for αλεx π ifesdjeen
3
330

Other Decks in Technology

See All in Technology
ZOZOのAI活用実践〜社内基盤からサービス応用まで〜
Avatar for ZOZO Developers zozotech
PRO
0
160
いま注目しているデータエンジニアリングの論点
Avatar for Ikki Miyazaki ikkimiyazaki
0
580
全てGoで作るP2P対戦ゲーム入門
Avatar for ponyo877 ponyo877
3
1.3k
[2025-09-30] Databricks Genie を利用した分析基盤とデータモデリングの IVRy の現在地
Avatar for 和田 悠佑 wxyzzz
0
450
LLMアプリケーション開発におけるセキュリティリスクと対策 / LLM Application Security
Avatar for GMO Flatt Security flatt_security
7
1.8k
Exadata Database Service on Dedicated Infrastructure(ExaDB-D) UI スクリーン・キャプチャ集
Avatar for oracle4engineer oracle4engineer
PRO
2
5.4k
AI Agentと MCP Serverで実現する iOSアプリの 自動テスト作成の効率化
Avatar for スパイダープラス株式会社 spiderplus_cb
0
470
成長自己責任時代のあるきかた/How to navigate the era of personal responsibility for growth
Avatar for Hiromu Shioya kwappa
3
250
自作LLM Native GORM Pluginで実現する AI Agentバックテスト基盤構築
Avatar for po3rin po3rin
2
240
Findy Team+のSOC2取得までの道のり
Avatar for adachi.ryo rvirus0817
0
310
業務自動化プラットフォーム Google Agentspace に入門してみる #devio2025
Avatar for maroon1st maroon1st
0
180
OpenAI gpt-oss ファインチューニング入門
Avatar for kmotohas kmotohas
2
910

Featured

See All Featured
No one is an island. Learnings from fostering a developers community.
Avatar for Antonio thoeni
21
3.5k
GraphQLの誤解/rethinking-graphql
Avatar for sonatard sonatard
73
11k
Facilitating Awesome Meetings
Avatar for Lara Hogan lara
56
6.6k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
Avatar for Irina Nazarova irinanazarova
9
960
Side Projects
Avatar for Sacha Greif sachag
455
43k
Designing Experiences People Love
Avatar for Jonathan Moore moore
142
24k
Navigating Team Friction
Avatar for Lara Hogan lara
189
15k
Reflections from 52 weeks, 52 projects
Avatar for Jefferson Lam jeffersonlam
352
21k
Imperfection Machines: The Place of Print at Facebook
Avatar for Scott Boms scottboms
269
13k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
Avatar for Dan Newman danielanewman
229
22k
Done Done
Avatar for chrislema chrislema
185
16k
Making Projects Easy
Avatar for Brett Harned brettharned
119
6.4k

Transcript

  1. Cascalog Hassle-free MapReduce that matches your scale Thursday, May 23,

    13

  2. Thursday, May 23, 13

  3. Setting expecations •This is not a guide •And not a

    tutorial •Doesn’t claim to be complete •Mostly to give you an idea •And encourage you to explore further Thursday, May 23, 13

  4. How much time do you spend on writing logic that

    framework should take care of? Thursday, May 23, 13

  5. How easy is it to debug your map/reduce aggragation? Thursday,

    May 23, 13

  6. Hadoop + Java composable, but too vebrose Pig, Hive too

    concrete, lack of abstraction and composition Thursday, May 23, 13

  7. Thursday, May 23, 13

  8. • Clear, declarative syntax • Inner and outer joins •

    Aggregators • Functions • Subqueries, composition • Sorting • Performant Thursday, May 23, 13

  9. Casca-WHAT? • Built on top of Hadoop (MapReduce) • Cascading

    (tuples, workflows, job execution) • Written in Clojure • Datalog (logic programming) Thursday, May 23, 13

  10. Abstract evrthn! Thursday, May 23, 13

  11. Source where data pours from Thursday, May 23, 13

  12. Pipe that data flows through Thursday, May 23, 13

  13. Filter that makes sure that only good stuff goes through

    Thursday, May 23, 13

  14. Tuple they actually flow Thursday, May 23, 13

  15. Thursday, May 23, 13

  16. Query anatomy Thursday, May 23, 13

  17. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Thursday, May 23, 13

  18. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Output Thursday, May 23, 13

  19. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    output vars Thursday, May 23, 13

  20. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Input input vars Thursday, May 23, 13

  21. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Logic/aggregations Thursday, May 23, 13

  22. Sources and sinks • HDFS (go figure) • Cassandra •

    MongoDB • SQL data sources • File system • Memory sources Thursday, May 23, 13

  23. (?<- (stdout) [?person] (age ?person 25)) Exact match of second

    element in a tuple Thursday, May 23, 13

  24. (defn younger-than? [limit age] (< age limit)) (?<- (stdout) [?person

    ?age] (age ?person ?age) (younger-than? 32 ?age)) Predicate match, fn call Predicate Thursday, May 23, 13

  25. (?<- (stdout) [?person ?count] (follows ?person _) (c/count ?count)) Aggregation

    Thursday, May 23, 13

  26. SHOWTIME! Thursday, May 23, 13

  27. Benefits •Query language is same as application language •Subqueries, reusability

    •Ad-hoc querying •Cascading underneath, so taps for all DBs work •Reuse application logic •Text editor integration Thursday, May 23, 13

  28. @ifesdjeen (twitter/github) Thursday, May 23, 13