Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems
Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
120
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
110
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
290
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
240
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
140
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
470
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
220

Other Decks in Technology

See All in Technology
DatabricksにOLTPデータベース『Lakebase』がやってきた！
Avatar for Takeru Ino inoutk
0
140
Lakebaseを使ったAIエージェントを実装してみる
Avatar for camay kameitomohiro
0
160
cdk initで生成されるあのファイル達は何なのか/cdk-init-generated-files
Avatar for tomoki10 tomoki10
0
230
オーティファイ会社紹介資料 / Autify Company Deck
Avatar for Autify autifyhq
10
130k
公開初日に Gemini CLI を試した話や FFmpeg と組み合わせてみた話など / Gemini CLI 初学者勉強会（#AI道場）
Avatar for you（@youtoy） you
PRO
0
210
american airlines®️ USA Contact Numbers: Complete 2025 Support Guide
Avatar for cassowary3740 supportflight
1
110
AIエージェントが書くのなら直接CloudFormationを書かせればいいじゃないですか何故AWS CDKを使う必要があるのさ
Avatar for watany watany
9
3.1k
United airlines®️ USA Contact Numbers: Complete 2025 Support Guide
Avatar for heron31238 unitedflyhelp
0
330
ABEMAの本番環境負荷試験への挑戦
Avatar for Miyazaki Taiga mk2taiga
5
350
スタートアップに選択肢を 〜生成AIを活用したセカンダリー事業への挑戦〜
Avatar for Nstock nstock
0
270
CDK Toolkit Libraryにおけるテストの考え方
Avatar for Makky12 smt7174
0
220
事例で学ぶ！B2B SaaSにおけるSREの実践例/SRE for B2B SaaS: A Real-World Case Study
Avatar for 株式会社ビットキー / Bitkey Inc. bitkey
1
170

Featured

See All Featured
Typedesign – Prime Four
Avatar for Hannes Fritz hannesfritz
42
2.7k
The Power of CSS Pseudo Elements
Avatar for Geoffrey Crofte geoffreycrofte
77
5.9k
Reflections from 52 weeks, 52 projects
Avatar for Jefferson Lam jeffersonlam
351
20k
How STYLIGHT went responsive
Avatar for nonsquared nonsquared
100
5.6k
Distributed Sagas: A Protocol for Coordinating Microservices
Avatar for Caitie McCaffrey caitiem20
331
22k
Unsuck your backbone
Avatar for Amy Palamountain ammeep
671
58k
Making the Leap to Tech Lead
Avatar for Ryan Cromwell cromwellryan
134
9.4k
Stop Working from a Prison Cell
Avatar for Chris Michel hatefulcrawdad
271
21k
Helping Users Find Their Own Way: Creating Modern Search Experiences
Avatar for Dan Newman danielanewman
29
2.7k
Save Time (by Creating Custom Rails Generators)
Avatar for Garrett Dimon garrettdimon
PRO
31
1.3k
Facilitating Awesome Meetings
Avatar for Lara Hogan lara
54
6.4k
jQuery: Nuts, Bolts and Bling
Avatar for Doug Neiner dougneiner
63
7.8k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com