Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
130
0

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
170
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
150
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
320
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
260
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
160
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
500
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
270

Other Decks in Technology

See All in Technology
12 個 Tool と、 4,096 Token
Avatar for ラワンアウンピョウ hlum
0
110
Google Cloud Next Tokyo 26登壇時のスクリプト
Avatar for Recruit recruitengineers
PRO
0
210
TinyGoでオブザーバビリティ？OpenTelemetryはTinyGoで使えるのか / Is OpenTelemetry available in TinyGo environment?
Avatar for ymotongpoo ymotongpoo
2
120
AIがコードを書く時代に、エンジニアは何で自分の価値を証明するのか / In an era where AI writes code how do engineers prove their value
Avatar for CARTA Engineering carta_engineering
1
660
「事実」を「計測」して、「解釈」する 〜AWS Lambdaのメモリ最適解の見つけ方〜
Avatar for maimyyym maimyyym
0
120
AIは爆速なのに、私が詰まっていた話 ― 音声入力と鳴くマスコットでボトルネックを削る
Avatar for Yuuki Yamashita yama3133
0
560
分析の民主化を支えるメタデータ整備の実践
Avatar for KAKEHASHI kakehashi
PRO
4
1.2k
パッチ更新でterraform initが落ちた日
Avatar for gr1m0h grimoh
1
100
AI時代のPRレビューにどう向き合うか？
Avatar for KairiM kairim0
1
110
「Docker Desktop不要の時代」は 来たのか？
Avatar for ueponx ueponx
0
340
ビジネスを止めない技術的負債の返済のための戦略とその手法 - 技術的負債と向き合う / Complexity and Simplicity
Avatar for soudai sone soudai
PRO
3
680
しろおびから始める！ログ活用の第一歩
Avatar for Shumei Ito shumei_ito
0
800

Featured

See All Featured
Digital Projects Gone Horribly Wrong (And the UX Pros Who Still Save the Day) - Dean Schuster
Avatar for UX Y'all uxyall
1
3.3k
The Curious Case for Waylosing
Avatar for Cassini Nazir cassininazir
1
560
The Organizational Zoo: Understanding Human Behavior Agility Through Metaphoric Constructive Conversations (based on the works of Arthur Shelley, Ph.D)
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
490
Neural Spatial Audio Processing for Sound Field Analysis and Control
Avatar for NII S. Koyama's Lab skoyamalab
0
550
Leading Effective Engineering Teams in the AI Era
Avatar for Addy Osmani addyosmani
9
2.7k
The Director’s Chair: Orchestrating AI for Truly Effective Learning
Avatar for Mike Taylor tmiket
1
320
Building a Modern Day 
E-commerce SEO Strategy
Avatar for Aleyda Solis aleyda
45
9.2k
Leveraging LLMs for student feedback in introductory data science courses - posit::conf(2025)
Avatar for Mine Cetinkaya-Rundel minecr
1
420
More Than Pixels: Becoming A User Experience Designer
Avatar for Michelle Schulp Hunt marktimemedia
3
560
Tell your own story through comics
Avatar for letsgokoyo letsgokoyo
1
1.1k
Keith and Marios Guide to Fast Websites
Avatar for Keith Pitt keithpitt
413
23k
Code Review Best Practice
Avatar for Trisha Gee trishagee
74
20k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com