Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
130
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
110
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
290
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
140
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
480
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
230

Other Decks in Technology

See All in Technology
CREが作る自己解決サイクルSlackワークフローに組み込んだAIによる社内ヘルプデスク改革 #cre_meetup
Avatar for 弁護士ドットコム bengo4com
0
320
まだ間に合う！ 2025年のhono/ssg事情
Avatar for watany watany
3
650
20251027_マルチエージェントとは
Avatar for Almondoイベント担当 almondo_event
0
300
だいたい分かった気になる 『SREの知識地図』 / introduction-to-sre-knowledge-map-book
Avatar for katsuhisa_ katsuhisa91
PRO
3
1.4k
IoTLT@ストラタシスジャパン_20251021
Avatar for Norippy norioikedo
0
100
生成AI時代のPythonセキュリティとガバナンス
Avatar for abenben abenben
0
130
データ戦略部門 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
1
3.8k
混合雲環境整合異質工作流程工具運行關鍵業務 Job 的經驗分享
Avatar for Yao-Siang Su (蘇曜祥) yaosiang
0
170
マルチエージェントのチームビルディング_2025-10-25
Avatar for shino shinoyamada
0
130
オブザーバビリティと育てた ID管理・認証認可基盤の歩み / The Journey of an ID Management, Authentication, and Authorization Platform Nurtured with Observability
Avatar for 株式会社カミナシ kaminashi
1
570
難しいセキュリティ用語をわかりやすくしてみた
Avatar for Yuta3110 yuta3110
0
390
NLPコロキウム20251022_超効率化への挑戦: LLM 1bit量子化のロードマップ
Avatar for Yuma Ichikawa yumaichikawa
2
420

Featured

See All Featured
Writing Fast Ruby
Avatar for Erik Berlin sferik
630
62k
What's in a price? How to price your products and services
Avatar for Michael Herold michaelherold
246
12k
Practical Tips for Bootstrapping Information Extraction Pipelines
Avatar for Matthew Honnibal honnibal
PRO
23
1.5k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
Avatar for Sam Stephenson sstephenson
162
15k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
Avatar for Joe Casabona jcasabona
34
2.5k
Automating Front-end Workflow
Avatar for Addy Osmani addyosmani
1371
200k
Designing for humans not robots
Avatar for Tammie Lister tammielis
254
26k
Producing Creativity
Avatar for Steve Smith orderedlist
PRO
347
40k
The Language of Interfaces
Avatar for Des Traynor destraynor
162
25k
The Straight Up "How To Draw Better" Workshop
Avatar for Dennis Kardys denniskardys
238
140k
CSS Pre-Processors: Stylus, Less & Sass
Avatar for Bermon Painter bermonpainter
359
30k
Fireside Chat
Avatar for paigeccino paigeccino
41
3.7k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com