Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
140
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
120
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
310
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
490
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
240

Other Decks in Technology

See All in Technology
スクリプトの先へ！AIエージェントと組み合わせる モバイルE2Eテスト
Avatar for Ryo WATANABE error96num
0
190
Claude Code 2026年 最新アップデート
Avatar for Oikon oikon48
14
11k
Cortex Code CLI と一緒に進めるAgentic Data Engineering
Avatar for あれ __allllllllez__
0
430
今のWordPress の制作手法ってなにがあんねん？(改) / What’s the Deal with WordPress Development These Days?
Avatar for tbshiki tbshiki
0
500
バクラク最古参プロダクトで重ねた技術投資を振り返る
Avatar for ypresto ypresto
0
170
OSC仙台プレ勉強会　AlmaLinuxとは
Avatar for koedoyoshida koedoyoshida
0
190
ガバメントクラウドにおけるAWSの長期継続割引について
Avatar for takeda_h takeda_h
2
5.3k
Oracle Cloud Infrastructure IaaS 新機能アップデート 2025/12 - 2026/2
Avatar for oracle4engineer oracle4engineer
PRO
0
170
AWS DevOps Agent vs SRE俺 / AWS DevOps Agent vs me, the SRE
Avatar for SMS tech sms_tech
3
900
詳解 強化学習 / In-depth Guide to Reinforcement Learning
Avatar for PRIN Lab prinlab
0
290
20260311 ビジネスSWG活動報告（デジタルアイデンティティ人材育成推進WG Ph2 活動報告会）
Avatar for OpenID Foundation Japan oidfj
0
350
GCASアップデート（202601-202603）
Avatar for 高橋広和 techniczna
0
220

Featured

See All Featured
Git: the NoSQL Database
Avatar for Brandon Keepers bkeepers
PRO
432
66k
Gemini Prompt Engineering: Practical Techniques for Tangible AI Outcomes
Avatar for Mfonobong Umondia mfonobong
2
320
Building Flexible Design Systems
Avatar for Yesenia Perez-Cruz yeseniaperezcruz
330
40k
The Hidden Cost of Media on the Web [PixelPalooza 2025]
Avatar for Tammy Everts tammyeverts
2
250
How to make the Groovebox
Avatar for あそなす asonas
2
2k
Neural Spatial Audio Processing for Sound Field Analysis and Control
Avatar for NII S. Koyama's Lab skoyamalab
0
220
Data-driven link building: lessons from a $708K investment (BrightonSEO talk)
Avatar for Szymon Słowik szymonslowik
1
980
SERP Conf. Vienna - Web Accessibility: Optimizing for Inclusivity and SEO
Avatar for Sara Fernández Carmona sarafernandez
1
1.3k
JAMstack: Web Apps at Ludicrous Speed - All Things Open 2022
Avatar for David Neal reverentgeek
1
390
A better future with KSS
Avatar for Kyle Neath kneath
240
18k
The Mindset for Success: Future Career Progression
Avatar for Greg Gifford greggifford
PRO
0
280
The Art of Delivering Value - GDevCon NA Keynote
Avatar for David Neal reverentgeek
16
1.9k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com