Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cascalog

Avatar for αλεx π αλεx π
May 22, 2013
Technology
160
4

 Cascalog

Short demo talk on Cascalog on Hadoop UG in Munich

Avatar for αλεx π

αλεx π

May 22, 2013

More Decks by αλεx π

See All by αλεx π
Scalable Time Series With Cassandra
Avatar for αλεx π ifesdjeen
1
420
Bayesian Inference is known to make machines biased
Avatar for αλεx π ifesdjeen
2
400
Cassandra for Data Analytics Backends
Avatar for αλεx π ifesdjeen
7
460
Stream Processing and Functional Programming
Avatar for αλεx π ifesdjeen
1
790
PolyConf 2015 - Rocking the Time Series boat with C, Haskell and ClojureScript
Avatar for αλεx π ifesdjeen
0
530
Clojure - A Sweetspot for Analytics
Avatar for αλεx π ifesdjeen
8
2.1k
Going Off Heap
Avatar for αλεx π ifesdjeen
3
1.9k
Always be learning
Avatar for αλεx π ifesdjeen
1
190
Learn Yourself Emacs For Great Good workshop slides
Avatar for αλεx π ifesdjeen
3
350

Other Decks in Technology

See All in Technology
[ChatGPT Work LT]事務作業が苦手な人のための バックオフィスの「半」自動化
Avatar for chimaki chimaki_iot
0
250
Invisible to AI? Making TYPO3 Sites Quotable by AI Search Systems
Avatar for Wolfgang Wagner wolfgangwagner
0
170
Pavlokで始める電撃駆動開発
Avatar for 佐藤陽威 sgrsn
0
180
Breaking the Seal: Static Deobfuscation of Compiled V8 JavaScript Bytecode Malware
Avatar for hasherezade hshrzd
0
670
Contract One Engineering Unit 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
0
19k
取引先から届く 「セキュリティチェックシート」の読み解き方
Avatar for かーでぃ kamadamakoto
0
140
Agent 時代の Kaggle 展望 / kaggle-in-the-agentic-era
Avatar for Shotaro Ishihara upura
1
680
Forza Horizon 6 のテレメトリ機能で 自動運転に使えそうな学習データを集める話
Avatar for henjin0 henjin0
0
150
ボトムアップ文化が強い組織で セキュリティをどう根付かせていくかの現在進行形の話 / Making Security Stick in a Bottom-Up Organization
Avatar for Takashi Yamaguchi yamaguchitk333
0
210
組織にどうSREを根付かせるか？〜IVRyの場合〜
Avatar for abnoumaru abnoumaru
0
330
LLM・AIエージェントシステムベストプラクティス
Avatar for shibuiwilliam shibuiwilliam
2
660
オートロックマンションなのに、各部屋は施錠なし！？ 攻撃者が組織内ネットワークで大暴れする理由 / The Front Door Is Locked, but the Rooms Are Wide Open: Why Attackers Move Freely Inside Enterprise Networks
Avatar for NTT docomo Business nttcom
1
1.6k

Featured

See All Featured
Taking LLMs out of the black box: A practical guide to human-in-the-loop distillation
Avatar for Ines Montani inesmontani
PRO
3
2.3k
The Art of Delivering Value - GDevCon NA Keynote
Avatar for David Neal reverentgeek
16
2.1k
Accessibility Awareness
Avatar for Sarah Abderemane sabderemane
1
170
New Earth Scene 8
Avatar for Sydney Stone popppiees
3
2.5k
Evolving SEO for Evolving Search Engines
Avatar for Ryan Jones ryanjones
0
250
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
Avatar for Addy Osmani addyosmani
10
1.3k
Amusing Abliteration
Avatar for ianozsvald ianozsvald
1
240
RailsConf 2023
Avatar for Aaron Patterson tenderlove
30
1.5k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
Avatar for Irina Nazarova irinanazarova
9
1.5k
Why You Should Never Use an ORM
Avatar for John Nunemaker jnunemaker
PRO
61
10k
Reality Check: Gamification 10 Years Later
Avatar for Sebastian Deterding codingconduct
0
2.2k
The agentic SEO stack - context over prompts
Avatar for schlessera schlessera
0
860

Transcript

  1. Cascalog Hassle-free MapReduce that matches your scale Thursday, May 23,

    13

  2. Thursday, May 23, 13

  3. Setting expecations •This is not a guide •And not a

    tutorial •Doesn’t claim to be complete •Mostly to give you an idea •And encourage you to explore further Thursday, May 23, 13

  4. How much time do you spend on writing logic that

    framework should take care of? Thursday, May 23, 13

  5. How easy is it to debug your map/reduce aggragation? Thursday,

    May 23, 13

  6. Hadoop + Java composable, but too vebrose Pig, Hive too

    concrete, lack of abstraction and composition Thursday, May 23, 13

  7. Thursday, May 23, 13

  8. • Clear, declarative syntax • Inner and outer joins •

    Aggregators • Functions • Subqueries, composition • Sorting • Performant Thursday, May 23, 13

  9. Casca-WHAT? • Built on top of Hadoop (MapReduce) • Cascading

    (tuples, workflows, job execution) • Written in Clojure • Datalog (logic programming) Thursday, May 23, 13

  10. Abstract evrthn! Thursday, May 23, 13

  11. Source where data pours from Thursday, May 23, 13

  12. Pipe that data flows through Thursday, May 23, 13

  13. Filter that makes sure that only good stuff goes through

    Thursday, May 23, 13

  14. Tuple they actually flow Thursday, May 23, 13

  15. Thursday, May 23, 13

  16. Query anatomy Thursday, May 23, 13

  17. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Thursday, May 23, 13

  18. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Output Thursday, May 23, 13

  19. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    output vars Thursday, May 23, 13

  20. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Input input vars Thursday, May 23, 13

  21. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Logic/aggregations Thursday, May 23, 13

  22. Sources and sinks • HDFS (go figure) • Cassandra •

    MongoDB • SQL data sources • File system • Memory sources Thursday, May 23, 13

  23. (?<- (stdout) [?person] (age ?person 25)) Exact match of second

    element in a tuple Thursday, May 23, 13

  24. (defn younger-than? [limit age] (< age limit)) (?<- (stdout) [?person

    ?age] (age ?person ?age) (younger-than? 32 ?age)) Predicate match, fn call Predicate Thursday, May 23, 13

  25. (?<- (stdout) [?person ?count] (follows ?person _) (c/count ?count)) Aggregation

    Thursday, May 23, 13

  26. SHOWTIME! Thursday, May 23, 13

  27. Benefits •Query language is same as application language •Subqueries, reusability

    •Ad-hoc querying •Cascading underneath, so taps for all DBs work •Reuse application logic •Text editor integration Thursday, May 23, 13

  28. @ifesdjeen (twitter/github) Thursday, May 23, 13