Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Cheating Your Way to Webscale
Search
Matt Robenolt
May 02, 2014
Programming
13
1.4k
Cheating Your Way to Webscale
Python Nordeste, May 2nd 2014
Matt Robenolt
May 02, 2014
Tweet
Share
More Decks by Matt Robenolt
See All by Matt Robenolt
Everything is broken and I don't know why.
mattrobenolt
0
44
I am bad at my job.
mattrobenolt
0
190
Everything is broken, and I don't know why. Python edition.
mattrobenolt
1
190
Everything is broken, and I don't know why. Python edition.
mattrobenolt
2
560
Varnish: How We Do It
mattrobenolt
1
210
Everything is broken, and I don't know why.
mattrobenolt
7
1.5k
HTTP for Great Good
mattrobenolt
85
200k
Caching is Hard: Varnish @ Disqus
mattrobenolt
52
2.1M
Developing & Deploying "Large" Scale Web Applications
mattrobenolt
25
1.3k
Other Decks in Programming
See All in Programming
Julia という言語について (FP in Julia « SIDE: F ») for 関数型まつり2025
antimon2
3
970
LINEヤフー データグループ紹介
lycorp_recruit_jp
0
760
データの民主化を支える、透明性のあるデータ利活用への挑戦 2025-06-25 Database Engineering Meetup#7
y_ken
0
270
Perplexity Slack Botを作ってAI活用を進めた話 / AI Engineering Summit プレイベント
n3xem
0
670
複数アプリケーションを育てていくための共通化戦略
irof
10
4k
Development of an App for Intuitive AI Learning - Blockly Summit 2025
teba_eleven
0
120
コードの90%をAIが書く世界で何が待っているのか / What awaits us in a world where 90% of the code is written by AI
rkaga
41
28k
C++20 射影変換
faithandbrave
0
500
Java on Azure で LangGraph!
kohei3110
0
160
Cursor Meetup Tokyo ゲノミクスとCursor: 進化と制約のあいだ
koido
2
1k
統一感のある Go コードを生成 AI の力で手にいれる
otakakot
0
3k
地方に住むエンジニアの残酷な現実とキャリア論
ichimichi
2
610
Featured
See All Featured
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
VelocityConf: Rendering Performance Case Studies
addyosmani
330
24k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
31
1.2k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
48
5.4k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.5k
KATA
mclloyd
29
14k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.8k
For a Future-Friendly Web
brad_frost
179
9.8k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
107
19k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
32
2.3k
Producing Creativity
orderedlist
PRO
346
40k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
124
52k
Transcript
Python Nordeste May 2nd 2014 Matt Robenolt Cheating Your Way
to #webscale
Hello < me irl
Lead Operations Engineer
Core Contributor
So what is #webscale?
10 million requests per second 4ms mean response time asynchronous
io mongodb
10 million requests per second 4ms mean response time asynchronous
io mongodb NOPE
Disqus only does 150 req/s per web server. * we
also write some bad code
150 12,960,000 388,800,000 per second per day per month real
world #webscale
Scale is about hiding the fact that your application is
actually really slow.
If your application feels fast, then it’s probably good enough.
Users hate waiting for shit.
So how do we do it?
Cheating 101
When a user asks for new data, let’s give them
old data instead.
When a user asks for new data, let’s give them
old data instead. Caching
When telling us to do something, let’s say we did
and maybe do it later.
When telling us to do something, let’s say we did
and maybe do it later. Queueing
Rule #1 Don’t get caught.
Rule #2 Don’t get caught.
Rule #3 Don’t get caught.
HTTP Caching
Introducing
tl;dr Varnish sits between your application and your users Internet
Let’s talk about HTTP. Hypertext Transport Protocol
$ curl -v disqus.com
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Request
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Method
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Path
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Version
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Headers
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Response
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Status
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Headers
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 For 300 seconds, all users will get the same response without talking to our application.
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 With power comes great responsibility.
GET / INTERNET Varnish Web servers
GET / INTERNET Varnish Web servers CACHED! “Cache-Control: max-age=300”
GET / INTERNET Varnish Web servers
GET / INTERNET Varnish Web servers CACHED!
BUT WAIT… THERE’S MORE
COLLAPSING REQUEST
GET / INTERNET Varnish Web servers
INTERNET Varnish Web servers GET /
INTERNET Varnish Web servers GET / If multiple users request
the same object, Varnish makes one fetch and returns to all users.
Queueing
Do as little work as possible, and return a promise
that this work will be done.
INTERNET Web servers Task workers Slow/Fast Data store Queue POST
/foo
INTERNET Web servers Task workers Slow/Fast Data store Queue POST
/foo
INTERNET Web servers Task workers Slow/Fast Data store Queue POST
/foo Workers can rate limit, debounce, increment counters, generate a fast materialized view, etc.
INTERNET Web servers Task workers Slow/Fast Data store Queue POST
/foo Make sure your tasks finish before a user tries to read the data back.
Final Thoughts
Understand your application. Where can you cheat without disrupting user
experience? Is seeing a few seconds old data going to damage a product?
Cheating should only enhance user experience.
Cheat any way you can, just don’t get caught.
Django & Varnish & RabbitMQ & Celery & PostgreSQL &
Redis & Cassandra & Riak Thanks
Questions? I have answers. ^ github.com/mattrobenolt @mattrobenolt some