Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
130
0

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
160
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
140
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
320
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
260
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
160
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
500
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
260

Other Decks in Technology

See All in Technology
会計事務所と顧問先の契約関係をOIDC・OAuthで表現する
Avatar for てらら terara
0
490
Claude Codeの体系的な理解と知識のフック
Avatar for Oikon oikon48
8
6k
Bill One 開発エンジニア 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
7
20k
アクセスキーが漏れた日にやるべきこと- 無効化の先にある本当の対応
Avatar for kazzpapa3 kazzpapa3
0
360
カーネルまで探検して理解するふたつの自動計装
Avatar for すみや sumiyae
0
340
MCPを待つな、パスキーを拡げよう（OAuth/OIDC Numa (Immersion) Workshop 2026）
Avatar for OpenID Foundation Japan oidfj
PRO
0
350
現場の暗黙知を継承するAIエージェント — 対話から生まれる長期記憶と Skills
Avatar for Atsuki Shirasawa atsukish
0
130
DMMブックスのNext.js化を加速させるAI活用 / Migrating DMM Books to Next.js with AI
Avatar for Kentaro Matsushita kentarom
0
180
1000⼈規模のClaude Enterprise運⽤を「Oktaのグループ」と「Slack」に集約する
Avatar for SansanTech sansantech
PRO
1
500
AWSとGitHub Actionsの責任境界と 組織で安全に使用する取り組み
Avatar for Nealle nealle
1
220
Data Hubグループ 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
0
3.2k
The Django UUID Story - DjangoCon US 2026
Avatar for Paolo Melchiorre pauloxnet
0
310

Featured

See All Featured
Technical Leadership for Architectural Decision Making
Avatar for Kenny Baas-Schwegler baasie
3
540
The B2B funnel & how to create a winning content strategy
Avatar for Katarina Dahlin katarinadahlin
PRO
1
490
GraphQLとの向き合い方2022年版
Avatar for Yosuke Kurami quramy
50
15k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
Avatar for Chris Coyier chriscoyier
508
140k
The Language of Interfaces
Avatar for Des Traynor destraynor
162
27k
Neural Spatial Audio Processing for Sound Field Analysis and Control
Avatar for NII S. Koyama's Lab skoyamalab
0
420
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
Avatar for panda_program panda_program
123
22k
What the history of the web can teach us about the future of AI
Avatar for Ines Montani inesmontani
PRO
1
670
Believing is Seeing
Avatar for Spiro Bolos oripsolob
1
200
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
Avatar for soudai sone soudai
PRO
201
75k
HTML-Aware ERB: The Path to Reactive Rendering @ RubyCon 2026, Rimini, Italy
Avatar for Marco Roth marcoroth
4
500
Building a Modern Day 
E-commerce SEO Strategy
Avatar for Aleyda Solis aleyda
45
9.2k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com