Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
130
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
110
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
290
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
240
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
140
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
480
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
230

Other Decks in Technology

See All in Technology
夢の印税生活 / Life on Royalties
Avatar for とみたまさひろ tmtms
0
280
AIドリブンのソフトウェア開発 - うまいやり方とまずいやり方
Avatar for Riotaro OKADA okdt
PRO
9
630
アジャイルテストで高品質のスプリントレビューを
Avatar for Shigeki Shoji takesection
0
120
事業価値と Engineering
Avatar for Recruit recruitengineers
PRO
2
230
JuniorからSeniorまで: DevOpsエンジニアの成長ロードマップ
Avatar for yuriemori yuriemori
0
220
[OCI Skill Mapping] AWSユーザーのためのOCI(2025年8月20日開催)
Avatar for oracle4engineer oracle4engineer
PRO
2
150
Goss: Faiss向けの新しい本番環境対応 Goバインディング #coefl_go_jp
Avatar for 弁護士ドットコム bengo4com
0
1.4k
Preferred Networks (PFN) とLLM Post-Training チームの紹介 / 第4回 関東Kaggler会 スポンサーセッション
Avatar for Preferred Networks pfn
PRO
1
230
実践アプリケーション設計 ①データモデルとドメインモデル
Avatar for Recruit recruitengineers
PRO
3
260
Postman MCP 関連機能アップデート / Postman MCP feature updates
Avatar for Yoichi Kawasaki yokawasa
0
150
開発と脆弱性と脆弱性診断についての話
Avatar for su3158 su3158
1
1.1k
MySQL HeatWave：サービス概要のご紹介
Avatar for oracle4engineer oracle4engineer
PRO
4
1.7k

Featured

See All Featured
How To Stay Up To Date on Web Technology
Avatar for Chris Coyier chriscoyier
790
250k
The Art of Programming - Codeland 2020
Avatar for Erika Heidi erikaheidi
55
13k
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
Avatar for Aki / @LoveIdahoBurger aki_iinuma
126
53k
Automating Front-end Workflow
Avatar for Addy Osmani addyosmani
1370
200k
Learning to Love Humans: Emotional Interface Design
Avatar for Aarron Walter aarron
273
40k
Done Done
Avatar for chrislema chrislema
185
16k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
Avatar for soudai sone soudai
PRO
183
54k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
Avatar for Marc Duiker marcduiker
31
2.2k
Git: the NoSQL Database
Avatar for Brandon Keepers bkeepers
PRO
431
65k
Code Reviewing Like a Champion
Avatar for maltzj maltzj
525
40k
YesSQL, Process and Tooling at Scale
Avatar for Rocio Delgado rocio
173
14k
Site-Speed That Sticks
Avatar for Harry Roberts csswizardry
10
790

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com