Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Base Lab Base Lab
May 07, 2015
Technology
120
0

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
160
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
140
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
320
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
260
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
160
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
500
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
260

Other Decks in Technology

See All in Technology
Terraform共通モジュールをチーム横断で“変えられる”運用へ ― リリースと適用の分離
Avatar for kekke-n kekke_n
1
3k
AI、CDK と協働する Full TypeScript アプリケーション開発 / Full TypeScript Application with AI and CDK
Avatar for ギークプラス ソフトウェア事業部 geekplus_tech
1
130
しくみを学んで使いこなそう GitHub Copilot app
Avatar for Toru Makabe torumakabe
2
250
関数型の考えを TypeScript に持ち込んで、テストしやすい純粋関数を増やす / Pure at the Core, Effects at the Edge: Bringing Functional Thinking into TypeScript
Avatar for 株式会社カミナシ kaminashi
1
120
世界、断片、モデル。そして理解
Avatar for SAKO Hiroshi ardbeg1958
1
120
知らん間に、回ってる
Avatar for ming ming_ayami
0
600
実装だけじゃない！ CCA-F取得エンジニアが教えるClaude Code開発プロセス活用術
Avatar for morimorikochan diggymo
2
730
AI Driven AI Governance
Avatar for Yutaka Hiroyama pict3
0
440
Claude Code公式skillで 自分の仕事を少しずつ手放そう！（Claude Code開発ノウハウ大公開スペシャル by クラスメソッド）
Avatar for Kei Masaoka kaym
1
350
Oracle Base Database Service 技術詳細
Avatar for oracle4engineer oracle4engineer
PRO
15
110k
ヘルスケア領域における AI 活用と その安全性担保のための取り組み (Leveraging AI in Healthcare and Our Efforts to Ensure Its Safety) - Google I/O Extended Tokyo 2026, July 11, 2026
Avatar for Akira Tameoka zettaittenani
0
380
Compose 新機能総まとめ / What's New in Jetpack Compose
Avatar for Yuki Anzai yanzm
0
260

Featured

See All Featured
Un-Boring Meetings
Avatar for Sebastian Deterding codingconduct
0
340
Producing Creativity
Avatar for Steve Smith orderedlist
PRO
348
40k
The Illustrated Children's Guide to Kubernetes
Avatar for Chris Short chrisshort
51
53k
Between Models and Reality
Avatar for mayunak mayunak
4
370
DevOps and Value Stream Thinking: Enabling flow, efficiency and business value
Avatar for Helen Beal helenjbeal
1
260
Java REST API Framework Comparison - PWX 2021
Avatar for Matt Raible mraible
34
9.5k
Dealing with People You Can't Stand - Big Design 2015
Avatar for Cassini Nazir cassininazir
367
27k
More Than Pixels: Becoming A User Experience Designer
Avatar for Michelle Schulp Hunt marktimemedia
3
460
Why Your Marketing Sucks and What You Can Do About It - Sophie Logan
Avatar for Sophie Logan marketingsoph
0
240
Navigating the moral maze — ethical principles for Al-driven product design
Avatar for Skipper Chong Warson skipperchong
2
410
The Organizational Zoo: Understanding Human Behavior Agility Through Metaphoric Constructive Conversations (based on the works of Arthur Shelley, Ph.D)
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
380
New Earth Scene 8
Avatar for Sydney Stone popppiees
3
2.4k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com