Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
140
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
120
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
300
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
490
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
240

Other Decks in Technology

See All in Technology
2026-02-24 月末 Tech Lunch Online #10 Cloud Runのデプロイの課題から考えるアプリとインフラの境界線
Avatar for SUZUKI Masashi masasuzu
0
100
Lookerの最新バージョンv26.2がやばい話
Avatar for Yuta Ozaki waiwai2111
1
140
Claude Codeはレガシー移行でどこまで使えるのか？
Avatar for ak2ie ak2ie
1
1.1k
WBCの解説は生成AIにやらせよう - 生成AIで野球解説者AI Agentを実現する / Baseball Commentator AI Agent for Gemini
Avatar for Shinichi Nakagawa shinyorke
PRO
0
290
【PyCon mini Shizuoka 2026】生成AI時代に画像処理やオーディオ処理のノードエディターを作る理由
Avatar for 高橋かずひと kazuhitotakahashi
0
180
AI が Approve する開発フロー / How AI Reviewers Accelerate Our Development
Avatar for Hiroka Zaitsu zaimy
1
230
AIエンジニア Devin と歩む、自律型運用プロセスの構築
Avatar for a2-ito a2ito
0
250
論文検索を日本語でできるアプリを作ってみた
Avatar for sai-lens sailen2
0
140
Introduction to Sansan, inc / Sansan Global Development Center, Inc.
Avatar for Sansan, Inc. sansan33
PRO
0
3k
Claude Codeと駆け抜ける 情報収集と実践録
Avatar for sontixyou sontixyou
2
1.2k
Webアクセシビリティ技術と実装の実際
Avatar for tomokusaba tomokusaba
0
140
OCI技術資料 : 外部接続 VPN接続 詳細
Avatar for Oracle Cloud Infrastructure ソリューション・エンジニア ocise
1
10k

Featured

See All Featured
WENDY [Excerpt]
Avatar for Tessa Abrams tessaabrams
9
36k
The Anti-SEO Checklist Checklist. Pubcon Cyber Week
Avatar for Ryan Jones ryanjones
0
82
WCS-LA-2024
Avatar for Leonardo Collado-Torres lcolladotor
0
470
Avoiding the “Bad Training, Faster” Trap in the Age of AI
Avatar for Mike Taylor tmiket
0
95
Rails Girls Zürich Keynote
Avatar for Gregor Martynus gr2m
96
14k
Automating Front-end Workflow
Avatar for Addy Osmani addyosmani
1371
200k
Ten Tips & Tricks for a 🌱 transition
Avatar for Manu Carrasco Molina stuffmc
0
82
Helping Users Find Their Own Way: Creating Modern Search Experiences
Avatar for Dan Newman danielanewman
31
3.1k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
Avatar for Michelle Schulp Hunt marktimemedia
31
2.7k
More Than Pixels: Becoming A User Experience Designer
Avatar for Michelle Schulp Hunt marktimemedia
3
340
Why Your Marketing Sucks and What You Can Do About It - Sophie Logan
Avatar for Sophie Logan marketingsoph
0
88
SERP Conf. Vienna - Web Accessibility: Optimizing for Inclusivity and SEO
Avatar for Sara Fernández Carmona sarafernandez
1
1.3k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com