Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
130
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
120
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
300
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
490
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
240

Other Decks in Technology

See All in Technology
名刺メーカーDevグループ 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
0
1k
小さく始めるBCP ― 多プロダクト環境で始める最初の一歩
Avatar for kekke-n kekke_n
1
380
Agile Leadership Summit Keynote 2026
Avatar for seki at druby.org m_seki
1
570
Contract One Engineering Unit 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
0
13k
CDK対応したAWS DevOps Agentを試そう_20260201
Avatar for モブエンジニア（Masaki Okuda） masakiokuda
1
240
Introduction to Sansan for Engineers / エンジニア向け会社紹介
Avatar for Sansan, Inc. sansan33
PRO
6
68k
生成AIを活用した音声文字起こしシステムの2つの構築パターンについて
Avatar for Kazuki Miura miu_crescent
PRO
2
180
AI駆動PjMの理想像 と現在地 -実践例を添えて-
Avatar for オカムラ masahiro_okamura
1
110
会社紹介資料 / Sansan Company Profile
Avatar for Sansan, Inc. sansan33
PRO
15
400k
What happened to RubyGems and what can we learn?
Avatar for Mike McQuaid mikemcquaid
0
270
Frontier Agents (Kiro autonomous agent / AWS Security Agent / AWS DevOps Agent) の紹介
Avatar for msysh msysh
3
160
2026年、サーバーレスの現在地 -「制約と戦う技術」から「当たり前の実行基盤」へ- /serverless2026
Avatar for Serverless Operations slsops
2
220

Featured

See All Featured
How to Talk to Developers About Accessibility
Avatar for Jason CranfordTeague jct
2
130
Jamie Indigo - Trashchat’s Guide to Black Boxes: Technical SEO Tactics for LLMs
Avatar for Tech SEO Connect techseoconnect
PRO
0
57
The Curse of the Amulet
Avatar for Matthew Lei leimatthew05
1
8.4k
The Language of Interfaces
Avatar for Des Traynor destraynor
162
26k
Avoiding the “Bad Training, Faster” Trap in the Age of AI
Avatar for Mike Taylor tmiket
0
75
Imperfection Machines: The Place of Print at Facebook
Avatar for Scott Boms scottboms
269
14k
The State of eCommerce SEO: How to Win in Today's Products SERPs - #SEOweek
Avatar for Aleyda Solis aleyda
2
9.5k
Introduction to Domain-Driven Design and Collaborative software design
Avatar for Kenny Baas-Schwegler baasie
1
580
Tell your own story through comics
Avatar for letsgokoyo letsgokoyo
1
810
Chasing Engaging Ingredients in Design
Avatar for Sebastian Deterding codingconduct
0
110
The Art of Delivering Value - GDevCon NA Keynote
Avatar for David Neal reverentgeek
16
1.8k
WCS-LA-2024
Avatar for Leonardo Collado-Torres lcolladotor
0
450

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com