Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
I am doing HTTP wrong
Search
Armin Ronacher
May 13, 2012
Programming
23
5.2k
I am doing HTTP wrong
A fresh look at HTTP for agile languages (more importantly: Python)
Armin Ronacher
May 13, 2012
Tweet
Share
More Decks by Armin Ronacher
See All by Armin Ronacher
Agentic Coding: The Future of Software Development with Agents
mitsuhiko
0
340
Do Dumb Things
mitsuhiko
0
750
No Assumptions
mitsuhiko
0
260
The Complexity Genie
mitsuhiko
0
250
The Catch in Rye: Seeding Change and Lessons Learned
mitsuhiko
0
360
Runtime Objects in Rust
mitsuhiko
0
360
Rust at Sentry
mitsuhiko
0
510
Overcoming Variable Payloads to Optimize for Performance
mitsuhiko
0
240
Rust API Design Learnings
mitsuhiko
0
600
Other Decks in Programming
See All in Programming
Things You Thought You Didn’t Need To Care About That Have a Big Impact On Your Job
hollycummins
0
210
dynamic!
moro
10
7.2k
Swift Concurrency - 状態監視の罠
objectiveaudio
2
510
ポスターセッション: 「まっすぐ行って、右!」って言ってラズパイカーを動かしたい 〜生成AI × Raspberry Pi Pico × Gradioの試作メモ〜
komofr
0
1.2k
組込みだけじゃない!TinyGo で始める無料クラウド開発入門
otakakot
0
210
株式会社 Sun terras カンパニーデック
sunterras
0
270
タスクの特性や不確実性に応じた最適な作業スタイルの選択(ペアプロ・モブプロ・ソロプロ)と実践 / Optimal Work Style Selection: Pair, Mob, or Solo Programming.
honyanya
3
160
『毎日の移動』を支えるGoバックエンド内製開発
yutautsugi
2
230
CSC305 Lecture 05
javiergs
PRO
0
210
大規模アプリのDIフレームワーク刷新戦略 ~過去最大規模の並行開発を止めずにアプリ全体に導入するまで~
mot_techtalk
0
430
AI Coding Meetup #3 - 導入セッション / ai-coding-meetup-3
izumin5210
0
2k
Building, Deploying, and Monitoring Ruby Web Applications with Falcon (Kaigi on Rails 2025)
ioquatix
4
1.9k
Featured
See All Featured
Bash Introduction
62gerente
615
210k
GitHub's CSS Performance
jonrohan
1032
470k
How to train your dragon (web standard)
notwaldorf
96
6.3k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
33
2.5k
The Power of CSS Pseudo Elements
geoffreycrofte
79
6k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
900
Facilitating Awesome Meetings
lara
56
6.6k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
32
2.3k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
285
14k
What's in a price? How to price your products and services
michaelherold
246
12k
Stop Working from a Prison Cell
hatefulcrawdad
271
21k
Testing 201, or: Great Expectations
jmmastey
45
7.7k
Transcript
I am doing HTTP wrong — a presentation by Armin
Ronacher @mitsuhiko
The Web developer's Evolution
echo
request.send_header(…) request.end_headers() request.write(…)
return Response(…)
Why Stop there?
What do we love about HTTP?
Text Based
REST
Cacheable
Content Negotiation
Well Supported
Works where TCP doesn't
Somewhat Simple
Upgrades to custom protocols
Why does my application look like HTTP?
everybody does it
Natural Conclusion
we can do better!
we're a level too low
Streaming: one piece at the time, constant memory usage, no
seeking.
Buffering: have some data in memory, variable memory usage, seeking.
TYPICAL Request / Response Cycle User Agent Proxy Server Application
Stream “Buffered” Dispatcher View
In Python Terms def application(environ, start_response): # Step 1: acquire
data data = environ['wsgi.input'].read(...) # Step 2: process data response = process_data(data) # Step 3: respond start_response('200 OK', [('Content-Type', 'text/plain')]) return [response]
One Level Up s = socket.accept() f = s.makefile('rb') requestline
= f.readline() headers = [] while 1: headerline = f.readline() if headerline == '\r\n': break headers.append(headerline)
Weird Mixture on the app request.headers <- buffered request.form <-
buffered request.files <- buffered to disk request.body <- streamed
HTTP's Limited signalling Strict Request / Response The only communication
during request from the server to the client is closing the connection once you started accepting the body.
Bailing out early def application(request): # At this point, headers
are parsed, everything else # is not parsed yet. if request.content_length > TWO_MEGABYTES: return error_response() ...
Bailing out a little bit later def application(request): # Read
a little bit of data request.input.read(4096) # You just committed to accepting data, now you have to # read everything or the browser will be very unhappy and # Just time out. No more responding with 413 ...
Rejecting Form fields -> memory File uploads -> disk What's
your limit? 16MB in total? All could go to memory. Reject file sizes individually? Needs overall check as well!
The Consequences How much data do you accept? Limit the
overall request size? Not helpful because all of it could be in-memory
It's not just limiting Consider a layered system How many
of you write code that streams? What happens if you pass streamed data through your layers?
A new approach
Dynamic typing made us lazy
we're trying to solve both use cases in one we're
not supporting either well
How we do it Hide HTTP from the apps HTTP
is an implementation detail
Pseudocode user_pagination = make_pagination_schema(User) @export( specs=[('page', types.Int32()), ('per_page', types.Int32())], returns=user_pagination,
semantics='select', http_path='/users/' ) def list_users(page, per_page): users = User.query.paginate(page, per_page) return users.to_dict()
Types are specific user_type = types.Object([ ('username', types.String(30)), ('email', types.Optional(types.String(250))),
('password_hash', types.String(250)), ('is_active', types.Boolean()), ('registration_date', types.DateTime()) ])
Why? Support for different input/output formats keyless transport support for
non-HTTP no hash collision attacks :-) Predictable memory usage
Comes for free Easier to test Helps documenting the public
APIs Catches common errors early Handle errors without invoking code Predictable dictionary ordering
Strict vs Lenient
Rule of Thumb Be strict in what you send, but
generous in what you receive — variant of Postel's Law
Being Generous In order to be generous you need to
know what to receive. Just accepting any input is a security disaster waiting to happen.
Support unsupported types { "foo": [1, 2, 3], "bar": {"key":
"value"}, "now": "Thu, 10 May 2012 14:16:09 GMT" } foo.0=1& foo.1=2& foo.2=3& bar.key=value& now=Thu%2C%2010%20May%202012%2014:16:09%20GMT
Solves the GET issue GET has no body parameters have
to be URL encoded inconsistency with JSON post requests
Where is the streaming?
There is none
there are always two sides to an API
If the server has streaming endpoints — the client will
have to support them as well
For things that need actual streaming we have separate endpoints.
streaming is different
but we can stream until we need buffering
Discard useless stuff { "foo": [list, of, thousands, of, items,
we don't, need], "an_important_key": "we're actually interested in" }
What if I don't make an API?
modern web apps are APIs
Dumb client? Move the client to the server
Q&A
Oh hai. We're hiring http://fireteam.net/careers