Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
130
0

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
160
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
140
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
320
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
260
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
160
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
500
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
260

Other Decks in Technology

See All in Technology
トヨタ⽣産⽅式(TPS)⼊⾨
Avatar for Recruit recruitengineers
PRO
3
650
Digitization部 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
2
7.7k
GitHub CopilotのFinOps - AI CreditのObservabilityと価値を生むためのエージェント設計
Avatar for yuriemori yuriemori
0
140
変化の早いClaude Codeを 書籍に落とし込む
Avatar for Oikon oikon48
7
1.3k
OSPN.JPバージョンアップ作業進捗のご報告 / 20260801-osc26kyoto
Avatar for Akira Ouchi akkiesoft
0
420
社内の7割が使うデータ基盤を、 データチーム2人で回すためにやったこと
Avatar for KOH koh_yoshi
4
1.2k
グローバル基準のSREは、運用現場でどう機能したか：成熟度アセスメントの実践 ／ SRE NEXT 2026
Avatar for sorawatanabe sorawatanabe
0
140
AI ネイティブな組織に Gemini Enterprise Agent Platform がなぜ必要なのか
Avatar for Asei Sugiyama asei
1
190
取引先から届く 「セキュリティチェックシート」の読み解き方
Avatar for かーでぃ kamadamakoto
0
140
【CEDEC2026】ゲームシナリオライターを支援するAIツール開発の実践 ― 設計とプロンプトの工夫 ―
Avatar for Cygames, Inc. cygames
PRO
1
770
Software Supply Chain Attackからクラウド環境を守るためにできること
Avatar for LHazy lhazy
2
220
Data Hubグループ 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
0
3.1k

Featured

See All Featured
The Hidden Cost of Media on the Web [PixelPalooza 2025]
Avatar for Tammy Everts tammyeverts
2
450
Noah Learner - AI + Me: how we built a GSC Bulk Export data pipeline
Avatar for Tech SEO Connect techseoconnect
PRO
0
350
Why Your Marketing Sucks and What You Can Do About It - Sophie Logan
Avatar for Sophie Logan marketingsoph
0
360
Automating Front-end Workflow
Avatar for Addy Osmani addyosmani
1370
210k
DevOps and Value Stream Thinking: Enabling flow, efficiency and business value
Avatar for Helen Beal helenjbeal
1
310
Effective software design: The role of men in debugging patriarchy in IT @ Voxxed Days AMS
Avatar for Kenny Baas-Schwegler baasie
0
470
Accessibility Awareness
Avatar for Sarah Abderemane sabderemane
1
170
A designer walks into a library…
Avatar for Paul Jervis Heath pauljervisheath
211
24k
Ecommerce SEO: The Keys for Success Now & Beyond - #SERPConf2024
Avatar for Aleyda Solis aleyda
1
2.1k
Rails Girls Zürich Keynote
Avatar for Gregor Martynus gr2m
96
14k
The MySQL Ecosystem @ GitHub 2015
Avatar for Sam Lambert samlambert
251
13k
The Impact of AI in SEO - AI Overviews June 2024 Edition
Avatar for Aleyda Solis aleyda
5
1.2k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com