Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
130
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
110
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
300
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
480
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
230

Other Decks in Technology

See All in Technology
Databricksによるエージェント構築
Avatar for Takaaki Yayoi taka_aki
1
140
32のキーワードで学ぶ はじめての耐量子暗号(PQC) / Getting Started with Post-Quantum Cryptography in 32 keywords
Avatar for quiver quiver
0
290
Kiro Autonomous AgentとKiro Powers の紹介 / kiro-autonomous-agent-and-powers
Avatar for tomoki10 tomoki10
0
160
Modern Data Stack大好きマンが語るSnowflakeの魅力
Avatar for Sagara sagara
0
300
意外とあった SQL Server 関連アップデート + Database Savings Plans
Avatar for Takuya Shibata stknohg
PRO
0
250
AI駆動開発によるDDDの実践
Avatar for ディップ株式会社 dip_tech
PRO
0
370
Bakuraku Engineering Team Deck
Avatar for LayerX layerx
PRO
12
6.6k
モバイルゲーム開発におけるエージェント技術活用への試行錯誤 ～開発効率化へのアプローチの紹介と未来に向けた展望～
Avatar for QualiArts qualiarts
0
530
セキュリティAIエージェントの現在と未来 / PSS #2 Takumi Session
Avatar for GMO Flatt Security flatt_security
3
1.5k
useEffectってなんで非推奨みたいなこと言われてるの？
Avatar for マグロ隊長kinTV maguroalternative
10
6.4k
Gemini でコードレビュー知見を見える化
Avatar for ZOZO Developers zozotech
PRO
1
130
生成AI時代の自動E2Eテスト運用とPlaywright実践知_引持力哉
Avatar for LegalOn Technologies, Inc legalontechnologies
PRO
0
190

Featured

See All Featured
Stop Working from a Prison Cell
Avatar for Chris Michel hatefulcrawdad
273
21k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
37
2.6k
ReactJS: Keep Simple. Everything can be a component!
Avatar for Pedro Nauck pedronauck
666
130k
Scaling GitHub
Avatar for Zach Holman holman
464
140k
StorybookのUI Testing Handbookを読んだ
Avatar for Shingo Yamazaki zakiyama
31
6.4k
Reflections from 52 weeks, 52 projects
Avatar for Jefferson Lam jeffersonlam
355
21k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
Avatar for panda_program panda_program
120
20k
Building Better People: How to give real-time feedback that sticks.
Avatar for Will Jessup wjessup
370
20k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
Avatar for Stéphanie Walter stephaniewalter
285
14k
4 Signs Your Business is Dying
Avatar for Josh Pigford shpigford
186
22k
Learning to Love Humans: Emotional Interface Design
Avatar for Aarron Walter aarron
274
41k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
48
9.8k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com