$30 off During Our Annual Pro Sale. View Details »

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
130
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
110
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
300
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
490
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
240

Other Decks in Technology

See All in Technology
Oracle Database@AWS：サービス概要のご紹介
Avatar for oracle4engineer oracle4engineer
PRO
1
410
[Neurogica] 採用ポジション/ Recruitment Position
Avatar for Neurogica neurogica
1
130
Authlete で実装する MCP OAuth 認可サーバー #CIMD の実装を添えて
Avatar for watahani watahani
0
180
Entity Framework Core におけるIN句クエリ最適化について
Avatar for tkym htkym
0
130
株式会社ビザスク_AI__Engineering_Summit_Tokyo_2025_登壇資料.pdf
Avatar for VISASQ eikohashiba
1
120
AI with TiDD
Avatar for Shiraji shiraji
1
300
SREが取り組むデプロイ高速化 ─ Docker Buildを最適化した話
Avatar for capytan capytan
0
150
マイクロサービスへの5年間 ぶっちゃけ何をしてどうなったか
Avatar for Tomohiro Hashidate joker1007
21
8.2k
AI駆動開発の実践とその未来
Avatar for Ikko Eltociear Ashimine eltociear
2
500
AWSの新機能をフル活用した「re:Inventエージェント」開発秘話
Avatar for みのるん minorun365
2
470
ソフトウェアエンジニアとAIエンジニアの役割分担についてのある事例
Avatar for KNOWLEDGE WORK / 株式会社ナレッジワーク kworkdev
PRO
0
280
通勤手当申請チェックエージェント開発のリアル
Avatar for WHIsaiyo whisaiyo
3
470

Featured

See All Featured
[SF Ruby Conf 2025] Rails X
Avatar for Vladimir Dementyev palkan
0
640
HDC tutorial
Avatar for Michiel Stock michielstock
0
280
From Legacy to Launchpad: Building Startup-Ready Communities
Avatar for Dug Song dugsong
0
110
What's in a price? How to price your products and services
Avatar for Michael Herold michaelherold
246
13k
How to train your dragon (web standard)
Avatar for Monica Dinculescu notwaldorf
97
6.5k
Building a A Zero-Code AI SEO Workflow
Avatar for Ian Lurie portentint
PRO
0
200
Documentation Writing (for coders)
Avatar for Carmen Chung carmenintech
77
5.2k
Sam Torres - BigQuery for SEOs
Avatar for Tech SEO Connect techseoconnect
PRO
0
150
Redefining SEO in the New Era of Traffic Generation
Avatar for Szymon szymonslowik
1
170
How to Talk to Developers About Accessibility
Avatar for Jason CranfordTeague jct
1
85
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
Avatar for soudai sone soudai
PRO
196
70k
Conquering PDFs: document understanding beyond plain text
Avatar for Ines Montani inesmontani
PRO
4
2.1k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com