Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cascalog

Avatar for αλεx π αλεx π
May 22, 2013
Technology
160
4

 Cascalog

Short demo talk on Cascalog on Hadoop UG in Munich

Avatar for αλεx π

αλεx π

May 22, 2013

More Decks by αλεx π

See All by αλεx π
Scalable Time Series With Cassandra
Avatar for αλεx π ifesdjeen
1
420
Bayesian Inference is known to make machines biased
Avatar for αλεx π ifesdjeen
2
400
Cassandra for Data Analytics Backends
Avatar for αλεx π ifesdjeen
7
460
Stream Processing and Functional Programming
Avatar for αλεx π ifesdjeen
1
790
PolyConf 2015 - Rocking the Time Series boat with C, Haskell and ClojureScript
Avatar for αλεx π ifesdjeen
0
530
Clojure - A Sweetspot for Analytics
Avatar for αλεx π ifesdjeen
8
2.1k
Going Off Heap
Avatar for αλεx π ifesdjeen
3
1.9k
Always be learning
Avatar for αλεx π ifesdjeen
1
190
Learn Yourself Emacs For Great Good workshop slides
Avatar for αλεx π ifesdjeen
3
350

Other Decks in Technology

See All in Technology
複数プロダクトで進めるAI機能実装 ── 実践から得たリアルな学びとロードマップ実現への挑戦 / AICon2026_yanari
Avatar for Rakus_Dev rakus_dev
0
130
プロダクト開発組織の現在地（Ver.2026/07） / product-organization
Avatar for 株式会社カオナビ kaonavi
0
120
そのドキュメント、自動化しませんか？
Avatar for yuk yuksew
1
310
LLM/Agent評価：トップ営業の発言を「正解」にする 〜暗黙的正解による評価を営業資産に変える〜
Avatar for Hirotaka takkuhiro
1
230
“それは自分の仕事じゃない"を 越えて行け
Avatar for yuukiyo yuukiyo
1
490
AICoEでAIネイティブ組織への進化
Avatar for YukiOgawa yukiogawa
0
200
Network Firewallやっていき！
Avatar for 荒木 ARK news_it_enj
0
170
CIで使うClaude
Avatar for TomoyaIwata iwatatomoya
0
290
「AIに依存している」と 「AIを使いこなしている」の違い
Avatar for Kensuke Yasuma k8yasuma
0
120
LLMやAIエージェントをソフトウェアに組み込むプラクティス
Avatar for shibuiwilliam shibuiwilliam
2
430
非定型なドキュメントを効率よくリファクタする 〜えぇ！？仕様書27本の移行が1日で終わったって！？〜
Avatar for subroh_0508 subroh0508
2
570
AI Native なプロダクト組織の立ち上げ方 : 生産性 100 倍への挑戦
Avatar for mikesorae mikesorae
0
170

Featured

See All Featured
Ethics towards AI in product and experience design
Avatar for Skipper Chong Warson skipperchong
2
330
Building a Modern Day 
E-commerce SEO Strategy
Avatar for Aleyda Solis aleyda
45
9.1k
Stewardship and Sustainability of Urban and Community Forests
Avatar for Eric Wiseman pwiseman
0
350
State of Search Keynote: SEO is Dead Long Live SEO
Avatar for Ryan Jones ryanjones
0
220
The Anti-SEO Checklist Checklist. Pubcon Cyber Week
Avatar for Ryan Jones ryanjones
0
180
Heart Work Chapter 1 - Part 1
Avatar for Lake Fama lfama
PRO
8
36k
GitHub's CSS Performance
Avatar for Jon Rohan jonrohan
1033
470k
The Limits of Empathy - UXLibs8
Avatar for Cassini Nazir cassininazir
1
510
How to audit for AI Accessibility on your Front & Back End
Avatar for davetheseo davetheseo
0
470
Build your cross-platform service in a week with App Engine
Avatar for Jose L jlugia
234
18k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
Avatar for Eileen M. Uchitelle eileencodes
141
35k
HU Berlin: Industrial-Strength Natural Language Processing with spaCy and Prodigy
Avatar for Ines Montani inesmontani
PRO
0
530

Transcript

  1. Cascalog Hassle-free MapReduce that matches your scale Thursday, May 23,

    13

  2. Thursday, May 23, 13

  3. Setting expecations •This is not a guide •And not a

    tutorial •Doesn’t claim to be complete •Mostly to give you an idea •And encourage you to explore further Thursday, May 23, 13

  4. How much time do you spend on writing logic that

    framework should take care of? Thursday, May 23, 13

  5. How easy is it to debug your map/reduce aggragation? Thursday,

    May 23, 13

  6. Hadoop + Java composable, but too vebrose Pig, Hive too

    concrete, lack of abstraction and composition Thursday, May 23, 13

  7. Thursday, May 23, 13

  8. • Clear, declarative syntax • Inner and outer joins •

    Aggregators • Functions • Subqueries, composition • Sorting • Performant Thursday, May 23, 13

  9. Casca-WHAT? • Built on top of Hadoop (MapReduce) • Cascading

    (tuples, workflows, job execution) • Written in Clojure • Datalog (logic programming) Thursday, May 23, 13

  10. Abstract evrthn! Thursday, May 23, 13

  11. Source where data pours from Thursday, May 23, 13

  12. Pipe that data flows through Thursday, May 23, 13

  13. Filter that makes sure that only good stuff goes through

    Thursday, May 23, 13

  14. Tuple they actually flow Thursday, May 23, 13

  15. Thursday, May 23, 13

  16. Query anatomy Thursday, May 23, 13

  17. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Thursday, May 23, 13

  18. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Output Thursday, May 23, 13

  19. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    output vars Thursday, May 23, 13

  20. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Input input vars Thursday, May 23, 13

  21. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Logic/aggregations Thursday, May 23, 13

  22. Sources and sinks • HDFS (go figure) • Cassandra •

    MongoDB • SQL data sources • File system • Memory sources Thursday, May 23, 13

  23. (?<- (stdout) [?person] (age ?person 25)) Exact match of second

    element in a tuple Thursday, May 23, 13

  24. (defn younger-than? [limit age] (< age limit)) (?<- (stdout) [?person

    ?age] (age ?person ?age) (younger-than? 32 ?age)) Predicate match, fn call Predicate Thursday, May 23, 13

  25. (?<- (stdout) [?person ?count] (follows ?person _) (c/count ?count)) Aggregation

    Thursday, May 23, 13

  26. SHOWTIME! Thursday, May 23, 13

  27. Benefits •Query language is same as application language •Subqueries, reusability

    •Ad-hoc querying •Cascading underneath, so taps for all DBs work •Reuse application logic •Text editor integration Thursday, May 23, 13

  28. @ifesdjeen (twitter/github) Thursday, May 23, 13