Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Szymon Sobczak - Hadoop + Storm
Search
Base Lab
May 07, 2015
Technology
0
93
Szymon Sobczak - Hadoop + Storm
Combo for realtime big data systems
Base Lab
May 07, 2015
Tweet
Share
More Decks by Base Lab
See All by Base Lab
Slawek Skowron - Monitoring @ Scale
baselab
0
110
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
baselab
0
100
Tomasz Nowak - Web Application Testing made easy
baselab
0
280
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
baselab
0
240
Mateusz Herych - LIKE '%smth%' is not the way
baselab
0
130
Jerzy Chałupski - Offline mode in Android apps
baselab
3
470
Jerzy Chałupski - Data model on Android
baselab
4
210
Other Decks in Technology
See All in Technology
eBPFのこれまでとこれから
yutarohayakawa
10
3.2k
チームビルディングは"感性"で向き合おう / Team Building with Awareness
kohzas
0
260
Analytics-Backed App Widget Development - Served with Jetpack Glance
miyabigouji
0
620
とあるOSSを継続可能にするための取り組みについて / OSS Refactoring Process
bun913
1
210
DevRelの始め方
moongift
PRO
2
390
内製化を目指す事業会社が、システム開発会社と共に進める「開発生産性改善」の取り組み事例 #devsumi
yuwji
1
110
AIで変わるテスト自動化:最新ツールの多様なアプローチ/ 20240910 Takahiro Kaneyama
shift_evolve
0
240
ロリポップ! for Gamersを支えるインフラ/lolipop for gamers infrastructure
takumakume
0
140
『GRANBLUE FANTASY: Relink』クオリティと物量の両立に挑戦したフェイシャルアニメーション事例 ~カットシーンからランタイムまで~
cygames
0
100
Google CloudのLLM活用の選択肢を広げるVertex AIのパートナーモデル
nayuts
0
130
【株式会社ELYZA】|GENIAC成果報告会 自社開発モデルプレゼンテーション
elyza
1
310
忙しい人のためのLangGraph概要まとめ
__ymgc__
1
190
Featured
See All Featured
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
278
13k
A Philosophy of Restraint
colly
202
16k
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
Fireside Chat
paigeccino
31
2.9k
Learning to Love Humans: Emotional Interface Design
aarron
270
40k
How To Stay Up To Date on Web Technology
chriscoyier
786
250k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
248
20k
Designing Experiences People Love
moore
138
23k
Designing for Performance
lara
604
68k
Automating Front-end Workflow
addyosmani
1365
200k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
123
18k
Music & Morning Musume
bryan
46
6k
Transcript
Szymon Sobczak
Hadoop + Storm Combo for realtime big data systems
Plan • Hadoop & Storm • Our setup • What
projects are we running • Decisions we had to make
Hadoop
None
Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,
“ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”
Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"
“ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”
Common traits
None
Base infrastructure services
Understand how the entire Base system works services
Big Data S3 uploader
Four example projects • Debugging • Reporting • Email intelligence
• Forecasting
Debugging S3 uploader
Reporting
Email analysis S3 uploader
Forecasting S3 uploader
Decisions we made ☑ Collect *all* data ☑ Put them
in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm
Summary S3 uploader
Questions?
Thank you
[email protected]
bigdata.getbase.com