Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
120
0

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
160
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
140
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
320
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
260
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
160
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
500
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
260

Other Decks in Technology

See All in Technology
PostgreSQL 19 新機能概要 OSC Hokkaido 2026
Avatar for Noriyoshi Shinoda nori_shinoda
0
180
フィジカル版Github Onshapeの紹介
Avatar for shiba_8ro shiba_8ro
0
290
AIはどのように 組織のアジリティを変えるのか？
Avatar for J.K junki
4
1.1k
Agent Skills設計で柔軟性と硬さのバランスが難しい話
Avatar for nassy nassy20
0
150
Chainlitで作るお手軽チャットUI
Avatar for tomo ynt0485
0
280
2026TECHFRESH畢業分享會 - AI 時代的人生存檔點
Avatar for LINE Developers Taiwan line_developers_tw
PRO
0
1.3k
ザ・データベース、MySQL ～ OSC 2026 Sendai ～
Avatar for sakaik sakaik
0
150
人材育成分科会.pdf
Avatar for _awache _awache
4
300
AIチャット検索改善の3週間
Avatar for KNOWLEDGE WORK / 株式会社ナレッジワーク kworkdev
PRO
2
140
AI時代のコスト管理を考えよう〜明日から使える実践AWSノウハウ～
Avatar for yoshimi0227 yoshimi0227
0
340
エラーバジェットのアラートのタイミングを考える.pdf
Avatar for KairiM kairim0
0
180
LayerX コーポレートエンジニアリング室におけるサプライチェーンセキュリティへの取り組み / Supply Chain Security at LayerX Corporate Engineering
Avatar for Yuya Takeyama yuyatakeyama
2
690

Featured

See All Featured
HDC tutorial
Avatar for Michiel Stock michielstock
2
720
The Organizational Zoo: Understanding Human Behavior Agility Through Metaphoric Constructive Conversations (based on the works of Arthur Shelley, Ph.D)
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
360
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
Avatar for Tammy Everts tammyeverts
11
950
Performance Is Good for Brains [We Love Speed 2024]
Avatar for Tammy Everts tammyeverts
12
1.7k
Faster Mobile Websites
Avatar for Dean Hume deanohume
310
31k
コードの90%をAIが書く世界で何が待っているのか / What awaits us in a world where 90% of the code is written by AI
Avatar for r-kagaya rkaga
62
44k
Marketing to machines
Avatar for Jono Alderson jonoalderson
1
5.5k
The Art of Programming - Codeland 2020
Avatar for Erika Heidi erikaheidi
57
14k
Game over? The fight for quality and originality in the time of robots
Avatar for Wayne Barker wayneb77
1
200
Reality Check: Gamification 10 Years Later
Avatar for Sebastian Deterding codingconduct
0
2.2k
Agile that works and the tools we love
Avatar for Rasmus Luckow-Nielsen rasmusluckow
331
21k
Winning Ecommerce Organic Search in an AI Era - #searchnstuff2025
Avatar for Aleyda Solis aleyda
1
2k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com