Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

Cascalog

Avatar for αλεx π αλεx π
May 22, 2013
Technology
170
4

 Cascalog

Short demo talk on Cascalog on Hadoop UG in Munich

Avatar for αλεx π

αλεx π

May 22, 2013

More Decks by αλεx π

See All by αλεx π
Scalable Time Series With Cassandra
Avatar for αλεx π ifesdjeen
1
430
Bayesian Inference is known to make machines biased
Avatar for αλεx π ifesdjeen
2
420
Cassandra for Data Analytics Backends
Avatar for αλεx π ifesdjeen
7
470
Stream Processing and Functional Programming
Avatar for αλεx π ifesdjeen
1
800
PolyConf 2015 - Rocking the Time Series boat with C, Haskell and ClojureScript
Avatar for αλεx π ifesdjeen
0
540
Clojure - A Sweetspot for Analytics
Avatar for αλεx π ifesdjeen
8
2.1k
Going Off Heap
Avatar for αλεx π ifesdjeen
3
1.9k
Always be learning
Avatar for αλεx π ifesdjeen
1
200
Learn Yourself Emacs For Great Good workshop slides
Avatar for αλεx π ifesdjeen
3
360

Other Decks in Technology

See All in Technology
AI × アーキテクト 第1・2章｜生成AIの課題とアーキテクトの素養
Avatar for Daisuke Motoki daidaidai
0
130
実践！ SRE × AI - ABEMA における AIOps -
Avatar for tetsuya28 tetsuya28
8
3.1k
AIは「聞けば答える」から「任せると自分で進める」へ - JACTIMペナン地域部会・ペナン三水会講演
Avatar for 西岡 賢一郎 (Kenichiro Nishioka) knishioka
0
110
Rustuna: Rustによる高速なOptuna実装
Avatar for Preferred Networks pfn
PRO
0
190
1つのリクエストを追って理解する - llm-dのP/D分離
Avatar for akhino akhino
1
150
日本語に強いオープンな大規模言語モデル「LLM-jp」
Avatar for Yusuke Oda odashi
4
1.1k
二歩目のフロントエンド・ウェブアクセシビリティ
Avatar for KAMIHARA Takumi camiha
2
530
Running Kotlin + Quarkus with Lambda
Avatar for Nealle nealle
1
3.5k
AIがコードを書く時代に、エンジニアは何で自分の価値を証明するのか / In an era where AI writes code how do engineers prove their value
Avatar for CARTA Engineering carta_engineering
3
1k
SAENM ~Jupyterで残すPoCの試行錯誤、未来の自分と仲間を助ける記録！~
Avatar for Manabu TERADA terapyon
0
110
「Docker Desktop不要の時代」は 来たのか？
Avatar for ueponx ueponx
0
730
遠藤な_めぐろLT #38 「社内のAI活用推進」@麻布台_20261009
Avatar for Naoya Endo endonanana
0
130

Featured

See All Featured
[RailsConf 2023 Opening Keynote] The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
31
10k
It's Worth the Effort
Avatar for 3n 3n
188
29k
The Invisible Side of Design
Avatar for Vitaly Friedman smashingmag
301
52k
The Anti-SEO Checklist Checklist. Pubcon Cyber Week
Avatar for Ryan Jones ryanjones
0
270
The Pragmatic Product Professional
Avatar for Laura Van Doore lauravandoore
37
7.5k
First, design no harm
Avatar for Per Axbom axbom
PRO
2
1.3k
Principles of Awesome APIs and How to Build Them.
Avatar for Keavy McMinn keavy
128
18k
The Language of Interfaces
Avatar for Des Traynor destraynor
162
27k
Digital Projects Gone Horribly Wrong (And the UX Pros Who Still Save the Day) - Dean Schuster
Avatar for UX Y'all uxyall
1
3.4k
How to Align SEO within the Product Triangle To Get 
Buy-In & Support - #RIMC
Avatar for Aleyda Solis aleyda
2
1.8k
The Organizational Zoo: Understanding Human Behavior Agility Through Metaphoric Constructive Conversations (based on the works of Arthur Shelley, Ph.D)
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
500
Design of three-dimensional binary manipulators for pick-and-place task avoiding obstacles (IECON2024)
Avatar for konakalab konakalab
0
620

Transcript

  1. Cascalog Hassle-free MapReduce that matches your scale Thursday, May 23,

    13

  2. Thursday, May 23, 13

  3. Setting expecations •This is not a guide •And not a

    tutorial •Doesn’t claim to be complete •Mostly to give you an idea •And encourage you to explore further Thursday, May 23, 13

  4. How much time do you spend on writing logic that

    framework should take care of? Thursday, May 23, 13

  5. How easy is it to debug your map/reduce aggragation? Thursday,

    May 23, 13

  6. Hadoop + Java composable, but too vebrose Pig, Hive too

    concrete, lack of abstraction and composition Thursday, May 23, 13

  7. Thursday, May 23, 13

  8. • Clear, declarative syntax • Inner and outer joins •

    Aggregators • Functions • Subqueries, composition • Sorting • Performant Thursday, May 23, 13

  9. Casca-WHAT? • Built on top of Hadoop (MapReduce) • Cascading

    (tuples, workflows, job execution) • Written in Clojure • Datalog (logic programming) Thursday, May 23, 13

  10. Abstract evrthn! Thursday, May 23, 13

  11. Source where data pours from Thursday, May 23, 13

  12. Pipe that data flows through Thursday, May 23, 13

  13. Filter that makes sure that only good stuff goes through

    Thursday, May 23, 13

  14. Tuple they actually flow Thursday, May 23, 13

  15. Thursday, May 23, 13

  16. Query anatomy Thursday, May 23, 13

  17. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Thursday, May 23, 13

  18. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Output Thursday, May 23, 13

  19. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    output vars Thursday, May 23, 13

  20. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Input input vars Thursday, May 23, 13

  21. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Logic/aggregations Thursday, May 23, 13

  22. Sources and sinks • HDFS (go figure) • Cassandra •

    MongoDB • SQL data sources • File system • Memory sources Thursday, May 23, 13

  23. (?<- (stdout) [?person] (age ?person 25)) Exact match of second

    element in a tuple Thursday, May 23, 13

  24. (defn younger-than? [limit age] (< age limit)) (?<- (stdout) [?person

    ?age] (age ?person ?age) (younger-than? 32 ?age)) Predicate match, fn call Predicate Thursday, May 23, 13

  25. (?<- (stdout) [?person ?count] (follows ?person _) (c/count ?count)) Aggregation

    Thursday, May 23, 13

  26. SHOWTIME! Thursday, May 23, 13

  27. Benefits •Query language is same as application language •Subqueries, reusability

    •Ad-hoc querying •Cascading underneath, so taps for all DBs work •Reuse application logic •Text editor integration Thursday, May 23, 13

  28. @ifesdjeen (twitter/github) Thursday, May 23, 13