Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Base Lab
May 07, 2015
Technology
0
94

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
baselab
0
110
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
baselab
0
110
Tomasz Nowak - Web Application Testing made easy
baselab
0
290
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
baselab
0
240
Mateusz Herych - LIKE '%smth%' is not the way
baselab
0
140
Jerzy Chałupski - Offline mode in Android apps
baselab
3
470
Jerzy Chałupski - Data model on Android
baselab
4
220

Other Decks in Technology

See All in Technology
なぜ私は自分が使わないサービスを作るのか？ / Why would I create a service that I would not use?
aiandrox
0
760
開発スピードは上がっている…品質はどうする？ スピードと品質を両立させるためのプロダクト開発の進め方とは #DevSumi #DevSumiB / Agile And Quality
nihonbuson
2
3k
次世代KYC活動報告 / 20250219-BizDay17-KYC-nextgen
oidfj
0
260
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
6
57k
ホワイトボードチャレンジ 説明&実行資料
ichimichi
0
130
デスクトップだけじゃないUbuntu
mtyshibata
0
170
個人開発から公式機能へ: PlaywrightとRailsをつなげた3年の軌跡
yusukeiwaki
11
3k
Nekko Cloud、 これまでとこれから ～学生サークルが作る、 小さなクラウド
logica0419
2
980
データマネジメントのトレードオフに立ち向かう
ikkimiyazaki
6
1k
目の前の仕事と向き合うことで成長できる - 仕事とスキルを広げる / Every little bit counts
soudai
25
7.2k
SA Night #2 FinatextのSA思想/SA Night #2 Finatext session
satoshiimai
1
140
インフラをつくるとはどういうことなのか、 あるいはPlatform Engineeringについて
nwiizo
5
2.6k

Featured

See All Featured
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.1k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
49
2.3k
The Pragmatic Product Professional
lauravandoore
32
6.4k
Side Projects
sachag
452
42k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
356
29k
The Art of Programming - Codeland 2020
erikaheidi
53
13k
How to train your dragon (web standard)
notwaldorf
91
5.8k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
40
2k
Adopting Sorbet at Scale
ufuk
74
9.2k
Six Lessons from altMBA
skipperchong
27
3.6k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com