Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
140
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
120
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
310
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
490
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
240

Other Decks in Technology

See All in Technology
コンテキスト・ハーネスエンジニアリングの現在
Avatar for Hirosato Gamo hirosatogamo
PRO
4
480
OSC仙台プレ勉強会　AlmaLinuxとは
Avatar for koedoyoshida koedoyoshida
0
190
WebアクセシビリティをCI/CDで担保する ― axe DevTools × Playwright C#実践ガイド
Avatar for tomokusaba tomokusaba
2
180
ソフトバンク流！プラットフォームエンジニアリング実現へのアプローチ
Avatar for SoftBank Tech Night sbtechnight
1
190
実践 Datadog MCP Server
Avatar for 株式会社ヌーラボ nulabinc
PRO
2
240
Goのerror型がシンプルであることの恩恵について理解する
Avatar for yamatai12 yamatai1212
1
210
ガバメントクラウドにおけるAWSの長期継続割引について
Avatar for takeda_h takeda_h
2
5.3k
最強のAIエージェントを諦めたら品質が上がった話 / how quality improved after giving up on the strongest AI agent
Avatar for kt2 kt2mikan
0
200
頼れる Agentic AI を支える Datadog のオブザーバビリティ / Powering Reliable Agentic AI with Datadog Observability
Avatar for Kento Kimura aoto
PRO
0
210
フロントエンド刷新 4年間の軌跡
Avatar for nus3 yotahada3
0
500
(Test) ai-meetup slide creation
Avatar for Oikon oikon48
3
450
AWS CDK「読めるけど書けない」を脱却するファーストステップ
Avatar for Makky12 smt7174
3
180

Featured

See All Featured
Evolving SEO for Evolving Search Engines
Avatar for Ryan Jones ryanjones
0
160
XXLCSS - How to scale CSS and keep your sanity
Avatar for Zaharenia Atzitzikaki sugarenia
249
1.3M
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
Avatar for Sam Stephenson sstephenson
162
16k
Designing Powerful Visuals for Engaging Learning
Avatar for Mike Taylor tmiket
0
290
How GitHub (no longer) Works
Avatar for Zach Holman holman
316
150k
How to Ace a Technical Interview
Avatar for Jacob Kaplan-Moss jacobian
281
24k
Bioeconomy Workshop: Dr. Julius Ecuru, Opportunities for a Bioeconomy in West Africa
Avatar for AKADEMIYA2063 akademiya2063
PRO
1
74
Building Flexible Design Systems
Avatar for Yesenia Perez-Cruz yeseniaperezcruz
330
40k
Lessons Learnt from Crawling 1000+ Websites
Avatar for Charles Meaden charlesmeaden
PRO
1
1.1k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
Avatar for Stéphanie Walter stephaniewalter
287
14k
Max Prin - Stacking Signals: How International SEO Comes Together (And Falls Apart)
Avatar for Tech SEO Connect techseoconnect
PRO
0
120
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
Avatar for panda_program panda_program
122
21k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com