Deep HTTP Dive Through Aleph & Netty

Learning HTTP protocol corners and implementation details from http://aleph.io and http://netty.org libraries.

Slides from at least 4-hours tech talk, so... there're quite a few of them :)

Transcript

1. Deep HTTP Dive Through Aleph & Netty Oleksii Kachaiev, @kachayev
2. @me • CTO at Attendify • 5+ years with Clojure in production • Creator of Muse | Aleph & Netty contributor • More: protocols, algebras, Haskell, Idris • @kachayev on Twitter & Github
4. The Goal • learn HTTP "corners" (a lot of them) • get familiar with Netty • take apart Aleph internals • all in parallel!
5. Prehistory Of The Talk • HTTP is a pretty complex • full picture = protocol + servers + clients + infrastructure • tons of implementation details
6. Agenda 1: Aleph • Netty overview • Aleph overview • HTTP server internals • HTTP client & connections handling internals
7. Agenda 2: Deep HTTP • connections: persistence, pipelining, idle • requests: multipart/form-data, 100 Continue • responses: chunked, compression • Upgrade: websocket • infranstructure: DNS, proxies • TLS, ALPN, HTTP/2
8. Aleph Is... • https://github.com/ztellman/aleph • "asynchronous communication for Clojure" © • netty + manifold + byte-streams + dirigiste
9. Hello World, Server (require '[aleph.http :as http]) (defn handler [req] {:status 200 :headers {"content-type" "text/plain"} :body "hello, world ! \n"}) (http/start-server handler {:port 8080})
10. Hello World, Client (require '[clojure.pprint :refer [pprint]]) (pprint @(http/get "http://localhost:8080/")) {:request-time 4, :aleph/keep-alive? true, :headers {"server" "Aleph/0.4.4", "content-type" "text/plain", "content-length" "18", "connection" "Keep-Alive", "date" "Sun, 08 Jul 2018 21:01:14 GMT"}, :status 200, :connection-time 0, :body #object[java.io.ByteArrayInputStream 0x69a5a174]}
11. Manifold
12. Manifold • https://github.com/ztellman/manifold • "a compatibility layer for event-driven abstractions" © • deferred: async value • d/chain, d/timeout!, d/onto • stream: ordered sequence of async values • in Aleph is used to wrap netty's channels • and to deal with streaming
13. Manifold: Basics (require '[manifold.deferred :as d]) (defn req->content-length [{:keys [headers]}] (Integer/parseInt (get headers "content-length" "0"))) (defn fetch-content-length [url] (d/chain' (http/get url) req->content-length)) (defn handler [req] (d/chain' (fetch-content-length "https://clojure.org") (fn [len] {:status 200 :headers {"content-type" "text/plain"} :body (str "content lenght is: " len \newline)})))
14. $ curl -v http://localhost:8080 * Rebuilt URL to: http://localhost:8080/ * Trying ::1... * TCP_NODELAY set * Connected to localhost (::1) port 8080 (#0) > GET / HTTP/1.1 > Host: localhost:8080 > User-Agent: curl/7.54.0 > Accept: */* > < HTTP/1.1 200 OK < Content-Type: text/plain < Server: Aleph/0.4.4 < Connection: Keep-Alive < Date: Mon, 09 Jul 2018 09:33:52 GMT < content-length: 25 < content lenght is: 20630 * Connection #0 to host localhost left intact
15. Byte-Streams
16. Byte-Streams • https://github.com/ztellman/byte-streams • "a rosetta stone for jvm byte representations" © • java.lang.String, byte arrays, byte buffers • *InputStream, *OutputStream, java.nio.channels.* • converts io.netty.buffer.*ByteBuf
17. Byte-Streams (d/chain' (http/get "http://localhost:8080") :body) ;; #object[java.io.ByteArrayInputStream 0x49f9d586] !
18. Byte-Streams (require '[byte-streams :as bs]) (d/chain' (http/get "http://localhost:8080") :body bs/to-string) ;; "content lenght is: 20630\n" !
19. Dirigiste
20. Dirigiste • https://github.com/ztellman/dirigiste • "centrally-planned object and thread pools" © • instrumented version of a j.u.c.ExecutorService • pluggable control mechanism to grow or shrink the pool • provides stats, e.g. QUEUE_LATENCY, TASK_ARRIVAL_RATE etc • tasks queue & j.u.c.RejectedExecutionException
21. Dirigiste • aleph.flow/instrumented-pool • max-queue-size is 65,536 by default • remember this ^ when planning backpressure • see more • "HTTP Server: Execution Flow" • "HTTP Client: Connections Pool"
22. Netty
23. Netty • https://github.com/netty/netty • "an event-driven asynchronous network application framework" © • biggest elephant in the room!
24. Netty: 40,000 Feets • events model & loops, native transports, non-blocking IO • communication design unification with Pipeline, Channel & Handler • a lot of ready-to-use handlers and codecs • ChannelFuture, ChannelPromise • *ByteBuf, zero-copy, smart allocations, leaks detector
25. Netty: How It Looks Like public class NettyExampleInitializer extends ChannelInitializer<Channel> { @Override public void initChannel(Channel ch) { ChannelPipeline pl = ch.pipeline(); pl.addLast(new SslHandler(...)); pl.addLast(new HttpServerCodec(...)); pl.addLast(new HttpContentCompressor(...)); pl.addLast(new ExampleRequestHandler(...)); pl.addLast(new IdleStateHandler(...)); } }
26. I/O Request via Channel | +---------------------------------------------------+---------------+ | ChannelPipeline | | | \|/ | | +----------------------------------------------+----------+ | | | IdleStateHanlder | | | +----------+-----------------------------------+----------+ | | /|\ | | | ........................................................... | | | \|/ | | +----------+-----------------------------------+----------+ | | | HttpServerCodec | | | +----------+-----------------------------------+----------+ | | /|\ | | | | \|/ | | +----------+-----------------------------------+----------+ | | | SslHandler | | | +----------+-----------------------------------+----------+ | | /|\ | | +---------------+-----------------------------------+---------------+ | \|/ +---------------+-----------------------------------+---------------+ | | | | | [ Socket.read() ] [ Socket.write() ] | | | | Netty Internal I/O Threads | +-------------------------------------------------------------------+
27. Netty: Inbound • fireChannelRegistered() • fireChannelActive() • fireChannelRead(Object) • fireChannelReadComplete() • fireExceptionCaught(Throwable) • fireUserEventTriggered(Object) • fireChannelWritabilityChanged()
28. Netty: Outbound • bind(SocketAddress, ChannelPromise) • connect(SocketAddress, SocketAddress, ChannelPromise) • write(Object, ChannelPromise) • flush() • read() • disconnect(ChannelPromise)
29. Netty ® Aleph • Netty is super cool, but kinda "low-level" • aleph.netty defines a lot of bridges • helpers to deal with ByteBufs • ChannelFuture → manifold's deferred • Channel represented as manifold's stream • a few macros to define ChannelHandlers • a lot more!
30. Aleph
31. Aleph: In Production (def s1 (http/start-server handler {:port 8080 :socket-address ... :bootstrap-transform ... :pipeline-transform ... :epoll? ... :executor ... :rejected-handler ... :ssl-context ... :max-header-size ... :max-chunk-size ... :compression-level 6 :idle-timeout ...}))
32. Aleph: In Production (defn fetch-content-length [url] (-> (http/get url {:pool ... :pool-timeout ... :connection-timeout ... :request-timeout ... :read-timeout ... :middleware ... :response-executor ...}) (d/chain' req->content-length) (d/catch' ...)))
33. Aleph: In Production • still a lot of corners • or details to take care about • a few config params (at least) to be aware of • will discuss them one by one as it goes
34. Code & Internals (40k Feets) • protocols & connections • execution model • http server • Netty bootstrap & pipeline • handlers, ring compatibility • http client & clj-http API
35. Aleph Server: Step ↺ Step • http/start-server delegates to http.server/start- server • setups executor • mind the defaults ! • delegates to netty/start-server • mind the on-close callback
36. http.server/start-server • detects epoll when necessary or uses NIO • defines SSL context injection • builds io.netty.bootstrap.ServerBootstrap • sets up chieldHandler to pipeline-initializer • binds to socket and waits for the Channel to be ready
37. aleph.http.server/start-server • reifies AlephServer • shutdown is not as trivial • java.io.Closeable • wait-for-close • on error
38. aleph.netty/pipeline-initializer • aleph.netty/pipeline-initializer • creates Netty's ChannelHandler that will • wait until Channel is registered on the Netty's Pipeline instance • call pipeline-builder provided as an argument • passing the instnace of Pipeline as an argument • clean up itself
39. Aleph: Waiting for The Channel I/O Request via Channel | +---------------------------------------------------+---------------+ | ChannelPipeline | | | \|/ | | +----------------------------------------------+----------+ | | | pipeline-initializer | | | +----------+-----------------------------------+----------+ | | /|\ | | +---------------+-----------------------------------+---------------+ | \|/ +---------------+-----------------------------------+---------------+ | | | | | [ Socket.read() ] [ Socket.write() ] | | | | Netty Internal I/O Threads | +-------------------------------------------------------------------+
40. aleph.http.server/pipeline- builder • pipeline-builder callback is defined here • sets up handlers, notably HttpServerCodec, HttpServerExpectContinueHandler • request handler is either ring-handler or raw-ring- handler • main task: read HTTP request and pass it to handle- request
41. I/O Request via Channel | +---------------------------------------------------+---------------+ | ChannelPipeline | | | | | | +--------------------------------------+ | | | | close-on-idle-handler (optional) | | | | +----------+---------------------------+ | | | /|\ \|/ | | +----------------------------------------------+----------+ | | | IdleStateHanlder (optional) | | | +----------+-----------------------------------+----------+ | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | ChunkedWriteHandler (optional) | | | +----------+-----------------------------------+----------+ | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | HttpContentCompressor (optional) | | | +----------+-----------------------------------+----------+ | | /|\ | | | +----------+---------------------------+ | | | | ring-handler or raw-ring-handler | | | | +----------+---------------------------+ | | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | HttpServerExpectContinueHandler | | | +----------+-----------------------------------+----------+ | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | HttpServerCodec | | | +----------+-----------------------------------+----------+ | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | SslHandler (optional) | | | +----------+-----------------------------------+----------+ | | /|\ | | +---------------+-----------------------------------+---------------+ | \|/ +---------------+-----------------------------------+---------------+ | | | | | [ Socket.read() ] [ Socket.write() ] | | | | Netty Internal I/O Threads | +-------------------------------------------------------------------+
42. aleph.http.server/handle-request • converts Netty's request → Ring-compatible request • runs handler (provided by the user) on a given executor • (or inlined!) • catches j.u.c.RejectedExecutionException and passes to rejected-handler • (by default answering with 503) • sends response when ready
43. Aleph Server: Step ↺ Step • pretty straightforward, right?
44. HTTP: The Idea
45. $ telnet 127.0.0.1 8080 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. GET / HTTP/1.1 Host: localhost:8080 HTTP/1.1 200 OK Content-Type: text/plain Server: Aleph/0.4.4 Connection: Keep-Alive Date: Sun, 08 Jul 2018 20:56:34 GMT content-length: 18 hello, world !
46. $ curl -v http://localhost:8080 * Rebuilt URL to: http://localhost:8080/ * Trying ::1... * TCP_NODELAY set * Connected to localhost (::1) port 8080 (#0) > GET / HTTP/1.1 > Host: localhost:8080 > User-Agent: curl/7.54.0 > Accept: */* > < HTTP/1.1 200 OK < Content-Type: text/plain < Server: Aleph/0.4.4 < Connection: Keep-Alive < Date: Sun, 08 Jul 2018 20:46:32 GMT < content-length: 18 < hello, world ! * Connection #0 to host localhost left intact
47. HTTP: The Idea • super simple! right?.. • RFC 2616 "Hypertext Transfer Protocol -- HTTP/1.1" • Obsoletes: 2068 • Updated by: 2817, 5785, 6266, 6585 • Obsoleted by: 7230, 7231, 7232, 7233, 7234, 7235 • RFC 7540 "Hypertext Transfer Protocol Version 2 (HTTP/2)" • ! have fun, guys!
48. HTTP/1.0 ↠ 1.1 • Host header • Persistent and pipelined connections • 100 Continue • Chunked & byte-range transfers • Compression & decompression • A lot more!
49. Host Header $ curl -v http://localhost:8080 > GET / HTTP/1.1 > Host: localhost:8080 ... • RFC 7230 #5.4 • ... a server MUST respond with a 400 ... • is not enforced neither by Aleph nor by Netty • it's not that practical nowadays...
50. Host Header $ telnet 127.0.0.1 8080 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. GET / HTTP/1.1 HTTP/1.1 200 OK Content-Type: text/plain ...
51. you SHOULD NOT rely on any RFCs — © The Reality
52. HTTP: Connections • "Keep-Alive" • connections pool • timeouts
53. HTTP: Persistent Connection • persistent connection a.k.a keep-alive • single TCP connection for multiple HTTP requests/ responses • HTTP/1.0 Connection: keep-alive • HTTP/1.1: all connections are persistent by default • HTTP/1.1 Connection: close when necessary
54. HTTP: Connections in Aleph • to reuse TCP connections Aleph uses pools • aleph.http/connection-pool builds flow/intrumented- pool • generate callback in the intrumented-pool creates a new connection • keep-alive? option is set to true by default
55. HTTP: Connections (def http10 (http/connection-pool {:connections-per-host 4 :total-connections 4 :max-queue-size 16 :connection-options {:keep-alive? false}})) (http/get "http://localhost:8080" {:pool http10}) {:request-time 29, :aleph/keep-alive? false, :headers {"server" "Aleph/0.4.4", "content-type" "text/plain", "content-length" "25", "connection" "Close", "date" "Mon, 09 Jul 2018 17:33:17 GMT"}, :status 200, :connection-time 6, :body #object[java.io.ByteArrayInputStream 0x75f16ce4]}
56. HTTP: Connections • on the client appropriate Connection: * header is set here • meaning you can mess this up a bit by setting header manually • Aleph server detects keep-alive "status" here and here • and uses here to send response • Aleph server adds the header automatically • ... still !
57. aleph.http.client/http-connection • aleph.http/create-connection delagates to aleph.http.client/http-connection • internal function, not a public API • connection is a function: request → deferred response • manifold streams to represent requests & responses (we can have many) • netty/create-client to build Netty's channel
58. aleph.http.client/http-connection • when channel is ready • defines a function to get request from public API • locks on channel to put! request and take! response • consumes requests one by one • ...
59. aleph.http.client/http-connection • builds HttpMessage & writes it to the channel • when response is ready checks errors • converts body to ByteArrayInputStream using buffer of a given size
60. HTTP Connections: Netty Channel • aleph.netty/create-client • creates io.netty.bootstrap.Bootstrap • set a few options: SO_REUSEADDR, MAX_MESSAGES_PER_READ • detects to use EpollEventLoopGroup or NioEventLoopGroup • sets handler to pipeline-initializer • connects to the remote address
61. Client: pipeline-initializer • same as for the server
62. Client: pipeline-builder • pipeline-builder argument for the initializer is defined here ! • updates Pipeline instance with a few new handlers, most notably: • HttpClientCodec with appropriate settings • "main" handler with Aleph's client logic • pipeline-transform option might be useful to rebuild Pipeline when necessary
63. Client: pipeline-builder • either raw-client-handler or client-handler depending on raw-stream? • raw-client-handler returns body as manifold's stream of Netty's ByteBuf • client-handler converts body to InputStream of bytes (additional copying but less frictions) • both implementations are kinda tricky • most of the complexity: buffering all the way down & chunks
64. I/O Request via Channel | +---------------------------------------------------+---------------+ | ChannelPipeline | | | | | | +--------------------------------------+ | | | | close-on-idle-handler (optional) | | | | +----------+---------------------------+ | | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | IdleStateHanlder (optional) | | | +----------+-----------------------------------+----------+ | | /|\ | | | +----------+---------------------------+ | | | | raw-client-handler or client-handler | | | | +----------+---------------------------+ | | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | HttpClientCodec | | | +----------+-----------------------------------+----------+ | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | SslHandler (optional) | | | +----------+-----------------------------------+----------+ | | /|\ | | | +----------+--------------------------+ | | | | pending-proxy-connection (optional) | | | | +----------+--------------------------+ | | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | proxy-handler (optional) | | | +----------+-----------------------------------+----------+ | | /|\ | | +---------------+-----------------------------------+---------------+ | \|/ +---------------+-----------------------------------+---------------+ | | | | | [ Socket.read() ] [ Socket.write() ] | | | | Netty Internal I/O Threads | +-------------------------------------------------------------------+
65. Connections Flow For The Public • aleph.http/request (rarely called directly) • "jumps" to the executor specified (or default-response- executor) • this might throw j.u.c.RejectedExecutionException • aleph.http/request is responsible for cleaning up after response is ready and on timeouts • also responsible for "top-level" middlewares: redirects & cookies
66. aleph.http/request • acquire connection from the pool specified (or default- connection-pool) • waits for the connection to be realized (either ready/reused or connecting) • "sends" the request applying connection function • chains on response and waits for :aleph/complete • disposes the connection from the pool when not keep-alive and on error
67. aleph.http/request • each stage tracks its own timeout since • PoolTimeout, ConnectionTimeout, RequestTimeout, ReadTimeout • never perform async operations w/o timeout • flexible error handling, easier to debug (reasoning is different) • you need this when implementing proxies or deciding on retries
68. HTTP: Connections... More! • idle cleanup • pipelining • debugging
69. HTTP Connections: Idle Timeout • persistent connection is meant to be "persistent" forever • not always the best option ! • idle-timeout option is available both for the client and the server since • when set, just updates the Pipeline builder • heavy lifting is done by Netty's IdleStateHandler • catching IdleStateEvent to close the connection
70. HTTP Connections: Pipelining • when multiple HTTP requests sent w/o waiting on responses • "allowed" with HTTP/1.1, not used widely (e.g. not used in modern browsers) • might dramatically reduce the number of TCP/IP packets • Aleph • supports pipelining on the server • does not support pipelining on the client
71. HTTP Connections: Pipelining (defn handler [{:keys [query-string]}] {:status 200 :body query-string}) (def s1 (http/start-server handler {:port 8072})) (-> (http/get "http://localhost:8072/?one") (deref 1e3) :body bs/to-string) ;; "one"
72. HTTP Connections: Pipelining (require '[aleph.tcp :as tcp]) (def tc @(tcp/client {:host "localhost" :port 8072})) ;; write 2 requests (s/put! tc (str "GET /?two HTTP/1.1\r\n\r\n" "GET /?three HTTP/1.1\r\n\r\n")) ;; read both responses (-> tc (s/take! ::closed) (deref 1e3) bs/to-string print)
73. HTTP Connections: Pipelining HTTP/1.1 200 OK Server: Aleph/0.4.4 Connection: Keep-Alive Date: Sat, 21 Jul 2018 14:12:24 GMT content-length: 3 twoHTTP/1.1 200 OK Server: Aleph/0.4.4 Connection: Keep-Alive Date: Sat, 21 Jul 2018 14:12:30 GMT content-length: 5 three
74. HTTP Connections: Pipelining • previous-response atom handles deferred • of the response that is currently processing • if any • next response is "scheduled" to be sent after
75. Debugging
76. HTTP Connections: Debugging • from time to time you need to trace what's going on with your connections • at least state changes: opened, closed, acquired, released • easiest way: inject a ChannelHandler that listens to all events and logs them • to catch acquire and release you need to wrap flow/ instrumented-pool
77. HTTP Connections: Debugging • clj-http has :debug and :save-request options • WIP to have similar functionality in Aleph
78. Takeaways
79. HTTP Connections: Takeaways • read the list of connection-pool options • mind the defaults • mind timeouts and exceptions • do not rely on the default pool • especially when sending a lot of requests
80. HTTP Request
81. Request & Response • ring req/resp are maps • io.netty.handler.codec.http provides • HttpRequest(s): DefaultHttpRequest, DefaultFullHttpRequest • HttpResponse(s): DefaultHttpResponse • also HttpMessage, HttpContent and more
82. Request & Response • Aleph defines intermediate representation • NettyRequest • NettyResponse • HeaderMap • "almost" maps with def-derived-map • "map type ... where key-value pairs may be derived from fields"
83. Headers • headers represented with "almost" map • mind that getting a header by the name is not taking element from a map • concatenates with "," in case we have few values • not always the best option • required by Ring Spec
84. HTTP: Multipart • Content-Type: multipart/form-data • boundary, content-disposition, content-type, content- transfer-encoding • long story in Aleph
85. HTTP: Multipart (def url "https://webhook.site/6b0e7099-0f8f-41e7-901d-54e678c349bc") (http/get url {:multipart [{:part-name "#1" :content "Hello"} {:part-name "#2" :content "World"}]})
86. HTTP: Multipart content-length 244 content-type multipart/form-data; boundary=1607b2e6cc0b5908 --1607b2e6cc0b5908 Content-Disposition: form-data; name="#1" Content-Type: application/octet-stream Hello --1607b2e6cc0b5908 Content-Disposition: form-data; name="#2" Content-Type: application/octet-stream World --1607b2e6cc0b5908--
87. HTTP: Multipart • just use :multipart instead of :body when sending request • generates random boundary, sets appropriate header • would be helpful to "remember" boundary generated in the request (e.g. for testing) !
88. HTTP: Multipart • follows clj-http :multipart format (at least, visually ) • clj-http uses org.apache.http.entity.mime.MultipartEntityBuilder • Aleph implements "from scratch" on the client • supported Content-Transfer-Encodings • no support for the server • yada's implementation with manifold's stream
89. 100 Continue
90. 100 Continue • client sends Expect: 100-continue and does not transmit body • server replies with status code 100 Continue or 417 Expectation failed • client send body • potentially, less pressure on the networks when sending large requests • rarely used in practise
91. $ curl -v -H "Expect: 100-continue" http://localhost:8080 * Rebuilt URL to: http://localhost:8080/ * Trying ::1... * TCP_NODELAY set * Connected to localhost (::1) port 8080 (#0) > GET / HTTP/1.1 > Host: localhost:8080 > User-Agent: curl/7.54.0 > Accept: */* > Expect: 100-continue > < HTTP/1.1 100 Continue < HTTP/1.1 200 OK < Content-Type: text/plain < Server: Aleph/0.4.4 < Connection: Keep-Alive < Date: Tue, 10 Jul 2018 12:38:15 GMT < content-length: 25 < content lenght is: 20630 * Connection #0 to host localhost left intact
92. 100 Continue • Aleph server support using HttpServerExpectContinueHandler • no built-in support for the client
93. HTTP: Chunked
94. HTTP: Chunked • not only for texts, body might be pretty large • Transfer-Encoding: chunked • each chunk: size, data, CRLF • last chunk: empty
95. HTTP: Chunked • RFC7230 #4.1 defines trailers & extensions, rarely used • RFC7230 #3.3.3 Transfer-Encoding suppresses Content- Length !
96. HTTP: Chunked • server: sending the response • server: reading the request • server: detecting last chunk of the request • client: reading the body • client: detecting last chunk • :max-chunk-size and :response-buffer-size options
97. aleph.http.core/send-message • good excuse to learn aleph.http.core/send-message • send-contiguous-body • send-file-body (send-chunked-file or send-file- region) • send-streaming-body • mind that try-set-content-length! is not called
98. HTTP: Iterator → Chunked (defn counting-handler [_] {:status 200 :headers {"content-type" "text/plain"} :body (map (partial str \newline) (range 5))}) (def sch (http/start-server counting-handler {:port 8094}))
99. HTTP: Iterator → Chunked $ curl -v http://localhost:8094 * Rebuilt URL to: http://localhost:8094/ * Trying ::1... * TCP_NODELAY set * Connected to localhost (::1) port 8094 (#0) > GET / HTTP/1.1 > Host: localhost:8094 > User-Agent: curl/7.54.0 > Accept: */* > < HTTP/1.1 200 OK < Content-Type: text/plain < Server: Aleph/0.4.4 < Connection: Keep-Alive < Date: Wed, 11 Jul 2018 15:34:43 GMT < transfer-encoding: chunked < 0 1 ...
100. HTTP: Iterator With Content- Length (defn counting-handler [_] {:status 200 :headers {"content-type" "text/plain" "content-length" "10"} :body (map (partial str \newline) (range 5))}) (def sch (http/start-server counting-handler {:port 8095}))
101. HTTP: Iterator With Content- Length $ curl -v http://localhost:8095 * Rebuilt URL to: http://localhost:8095/ * Trying ::1... * TCP_NODELAY set * Connected to localhost (::1) port 8095 (#0) > GET / HTTP/1.1 > Host: localhost:8095 > User-Agent: curl/7.54.0 > Accept: */* > < HTTP/1.1 200 OK < Content-Type: text/plain < Content-Length: 10 < Server: Aleph/0.4.4 < Connection: Keep-Alive < Date: Thu, 12 Jul 2018 07:03:26 GMT < 0 1
102. HTTP: Chunked & Aleph • Aleph responses with "Transter-Encoding: chunked" when :body is seq, iterator or stream • if Content-Lenght header is not set explicitely • detection client disconnect is still kinda tough • think about buffering and throttling in advance, this talk might help
103. HTTP: Compression
104. HTTP: Compression Headers • HTTP/1.1 headers • Accept-Encoding for the client • Content-Encoding for the server (whole connection) • Transfer-Encoding for the server (hop-by-hop)
105. HTTP: Compression Headers $ curl -v -H "Accept-Encoding: gzip" https://www.apple.com/ * Trying 2.17.173.185... * TCP_NODELAY set * Connected to www.apple.com (2.17.173.185) port 443 (#0) ... > GET / HTTP/1.1 > Host: www.apple.com > User-Agent: curl/7.54.0 > Accept: */* > Accept-Encoding: gzip > < HTTP/1.1 200 OK < Server: Apache < Content-Type: text/html; charset=UTF-8 ... < Content-Encoding: gzip ... < Content-Length: 7108 < Connection: keep-alive
106. HTTP: Compression Algorithms • compress - Lempel-Ziv-Welch (LZW) algorithm • deflate - deflate algorithm, RFC 1951 • gzip - Lempel-Ziv coding (LZ77) • identity • br - Brotli Compressed Data Format, IETF
107. HTTP: Compression (def s2 (http/start-server handler {:port 8083 :compression? true :compression-level 4}))
108. HTTP: Compression $ curl -v http://localhost:8083 * Rebuilt URL to: http://localhost:8083/ * Trying ::1... * TCP_NODELAY set * Connected to localhost (::1) port 8083 (#0) > GET / HTTP/1.1 > Host: localhost:8083 > User-Agent: curl/7.54.0 > Accept: */* > < HTTP/1.1 200 OK < Content-Type: text/plain < Server: Aleph/0.4.4 < Connection: Keep-Alive < Date: Tue, 10 Jul 2018 04:47:03 GMT < content-length: 25 < content lenght is: 20630 * Connection #0 to host localhost left intact
109. HTTP: Compression $ curl -v -H "Accept-Encoding: gzip" http://localhost:8083 * Rebuilt URL to: http://localhost:8083/ * Trying ::1... * TCP_NODELAY set * Connected to localhost (::1) port 8083 (#0) > GET / HTTP/1.1 > Host: localhost:8083 > User-Agent: curl/7.54.0 > Accept: */* > Accept-Encoding: gzip > < HTTP/1.1 200 OK < Content-Type: text/plain < Server: Aleph/0.4.4 < Connection: Keep-Alive < Date: Tue, 10 Jul 2018 04:48:21 GMT < content-encoding: gzip < transfer-encoding: chunked < * Connection #0 to host localhost left intact J+IQQR02036
110. HTTP: Compression • compression is disabled by default • supports custom compression level since • heavy lifting is done by io.netty.handler.codec.http.HttpContentCompressor
111. HTTP: Compression In Netty HttpContentCompressor ↳ HttpContentEncoder ... ↳ MessageToMessageCodec<HttpRequest,HttpObject> ... ... ↳ ChannelDuplexHandler • supports gzip or deflate, Brotli is an open question • respects Accept-Encoding header value
112. HTTP: Compression • mind the instance of io.netty.handler.stream.ChunkedWriteHandler • this forces send-file-body to use send-chunked-file instead of send-file-region • why? send-file-region uses zero-copy file transfer with io.netty.channel.DefaultFileRegion • does not support user-space modifications, e.g. compression !
113. HTTP: WebSocket
114. HTTP: WebSocket • full-duplex communication • RFC 6455 "The WebSocket Protocol" • handshaking using HTTP Upgrade header (compatibility) • Aleph uses manifold's SplicedStream to represent duplex channel • supports Text and Binary frames, replies to Ping frames • a lot of cases and corners in the protocol (duplex communication is hard)
115. HTTP: WebSocket Upgrade $ curl -v -H "Upgrade: websocket" \ -H "Connection: upgrade" \ -H "Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw==" \ -H "Sec-WebSocket-Protocol: chat" \ -H "Sec-WebSocket-Version: 13" \ http://echo.websocket.org * Rebuilt URL to: http://echo.websocket.org/ * Trying 174.129.224.73... * TCP_NODELAY set * Connected to echo.websocket.org (174.129.224.73) port 80 (#0) > GET / HTTP/1.1 > Host: echo.websocket.org > Upgrade: websocket > Connection: upgrade > Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw== > Sec-WebSocket-Protocol: chat > Sec-WebSocket-Version: 13
116. HTTP: WebSocket Upgrade < < HTTP/1.1 101 Web Socket Protocol Handshake < Connection: Upgrade < Date: Tue, 10 Jul 2018 09:11:33 GMT < Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk= < Server: Kaazing Gateway < Upgrade: websocket <
117. HTTP: Aleph WebSocket Client (require '[manifold.stream :as s]) (def ws @(http/websocket-client "wss://echo.websocket.org")) (class ws) ;; manifold.stream.SplicedStream (s/put! ws "Hello, world =)") ;; true (s/take! ws) ;; "Hello, world =)"
118. HTTP: Aleph WebSocket Client • http/websocket-client delegates to aleph.http.client/websocket-connection • mind the difference with aleph.http/websocket- connection ! • http.client/websocket-connection builds a Channel with netty/create-client • websocket-client-handler creates a duplex stream and a handler
119. I/O Request via Channel | +---------------------------------------------------+---------------+ | ChannelPipeline | | | | | | +--------------------------------------+ | | | | websocket-client-handler | | | | +----------+---------------------------+ | | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | WebSocketClientExtensionHandler (optional) | | | +----------+-----------------------------------+----------+ | | /|\ | | | +----------+---------------------------+ | | | | WebSocketFrameAggregator | | | | +----------+---------------------------+ | | | /|\ | | | +----------+---------------------------+ | | | | HttpObjectAggregator | | | | +----------+---------------------------+ | | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | HttpClientCodec | | | +----------+-----------------------------------+----------+ | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | SslHandler (optional) | | | +----------+-----------------------------------+----------+ | | /|\ | | +---------------+-----------------------------------+---------------+ | \|/ +---------------+-----------------------------------+---------------+ | | | | | [ Socket.read() ] [ Socket.write() ] | | | | Netty Internal I/O Threads | +-------------------------------------------------------------------+
120. HTTP: Aleph WebSocket Client • handler is responsible for • a handshake processing • Close frame sending • reacting appropriatly on incoming frames • non-websocket data payload throws an IllegalStateException
121. HTTP: Aleph WebSocket Server (defn ws-handler [req] (let [ws-conn @(http/websocket-connection req)] (s/connect (s/map #(str "echo: " %) ws-conn) ws-conn))) (def ws-server (http/start-server ws-handler {:port 8086})) (def ws @(http/websocket-client "ws://localhost:8086")) (s/put! ws "heh") ;; true (s/take! ws) ;; "echo: heh"
122. HTTP: Aleph WebSocket Server • not very intuitive ! • the idea here is to process HTTP/1.1 request first and then perform "upgrade" • http.server/websocket-upgrade-request? might be useful to "test" the request
123. HTTP: Aleph WebSocket Server • http/websocket-connection takes request and delegates to http.server/initialize-websocket-handler • initialize-websocket-handler builds and runs handshaker • .websocket? mark is set to modify response sending behavior • Pipeline is rebuilt appropriately • 2 streams spliced into one, as for the client
124. I/O Request via Channel | +---------------------------------------------------+---------------+ | ChannelPipeline | | | | | | +--------------------------------------+ | | | | websocket-server-handler | | | | +----------+---------------------------+ | | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | WebSocketServerCompressionHandler (optional) | | | +----------+-----------------------------------+----------+ | | /|\ | | | +----------+---------------------------+ | | | | WebSocketFrameAggregator | | | | +----------+---------------------------+ | | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | ChunkedWriteHandler (optional) | | | +----------+-----------------------------------+----------+ | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | HttpContentCompressor (optional) | | | +----------+-----------------------------------+----------+ | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | HttpServerCodec | | | +----------+-----------------------------------+----------+ | | /|\ \|/ | | +----------+-----------------------------------+----------+ | | | SslHandler (optional) | | | +----------+-----------------------------------+----------+ | | /|\ | | +---------------+-----------------------------------+---------------+ | \|/ +---------------+-----------------------------------+---------------+ | | | | | [ Socket.read() ] [ Socket.write() ] | | | | Netty Internal I/O Threads | +-------------------------------------------------------------------+
125. HTTP: Aleph & WebSocket • seq of the "connection close" events is "almost RFC" • client sends CloseFrame before closing the connection • on receiving CloseFrame saves status & reason • server sends CloseFrame w/o closing the connection • as it will be done by Netty • Netty behavior is "more RFC-ish"
126. HTTP: Aleph & WebSocket • client and server support permessage-deflate extension since • fine-grained Ping/Pong support is still an open question • to add ability to send http/websocket-ping manually, and to wait for Pong • helpful for heartbeats, online presence detection etc • pipeline-transform might be used to extend both server and client
127. DNS
128. DNS $ curl -v https://github.com/ * Trying 192.30.253.113... * TCP_NODELAY set * Connected to github.com (192.30.253.113) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 ... > GET / HTTP/1.1 > Host: github.com > User-Agent: curl/7.54.0 > Accept: */* > < HTTP/1.1 200 OK < Server: GitHub.com
129. DNS • where did we get 192.30.253.113? • DNS is a huge topic • JDK provides built-in mechanism with java.net.InetAddress
130. DNS (import 'java.net.InetAddress) (InetAddress/getByName "github.com") ;; #object[java.net.Inet4Address 0x471abb1c "github.com/192.30.253.113"] • blocking ! • impossible to employ an alternative cache/retry policy • no way to use alternative name resolution
131. DNS • Aleph supports pluggable name resolver since • when :dns-options specified, sets up :name-resolver to async DNS resolver (def dns-pool (http/connection-pool {:dns-options {:name-servers ["8.8.4.4"]}})) (d/chain' (http/get "https://github.com/" {:pool dns-pool}) :status)
132. DNS • heavy lifting is done by io.netty.resolver.dns.DnsAddressResolverGroup • Aleph's part is mostly params juggling • supports epoll detection • and flexible configuration format for name server providers • aleph.http/create-connection uses InetSocketAddress/createUnresolved
133. DNS • still relies on aleph.utils.PluggableDnsAddressResolverGroup as a workaround • going to clean this up since Netty #7793 was merged • ... done
134. HTTP Proxy
135. HTTP Proxy $ curl -v -x http://103.43.40.96:8080 http://netty.io/ * Trying 103.43.40.96... * TCP_NODELAY set * Connected to 103.43.40.96 (103.43.40.96) port 8080 (#0) > GET http://netty.io/ HTTP/1.1 > Host: netty.io > User-Agent: curl/7.54.0 > Accept: */* > Proxy-Connection: Keep-Alive > < HTTP/1.1 200 OK < Date: Tue, 10 Jul 2018 10:59:43 GMT
136. HTTP Tunneling Proxy: CONNECT $ curl -v -x http://103.43.40.96:8080 --proxytunnel http://netty.io/ * Trying 103.43.40.96... * TCP_NODELAY set * Connected to 103.43.40.96 (103.43.40.96) port 8080 (#0) * Establish HTTP proxy tunnel to netty.io:80 > CONNECT netty.io:80 HTTP/1.1 > Host: netty.io:80 > User-Agent: curl/7.54.0 > Proxy-Connection: Keep-Alive > < HTTP/1.1 200 OK < * Proxy replied OK to CONNECT request > GET / HTTP/1.1 > Host: netty.io > User-Agent: curl/7.54.0 > Accept: */* > < HTTP/1.1 200 OK < Date: Tue, 10 Jul 2018 11:00:57 GMT
137. HTTP Proxy: HTTPS $ curl -v -x http://103.43.40.96:8080 https://github.com * Rebuilt URL to: https://github.com/ * Trying 103.43.40.96... * TCP_NODELAY set * Connected to 103.43.40.96 (103.43.40.96) port 8080 (#0) * Establish HTTP proxy tunnel to github.com:443 > CONNECT github.com:443 HTTP/1.1 > Host: github.com:443 > User-Agent: curl/7.54.0 > Proxy-Connection: Keep-Alive > < HTTP/1.1 200 OK < * Proxy replied OK to CONNECT request * ALPN, offering h2 * ALPN, offering http/1.1 ... > GET / HTTP/1.1 > Host: github.com > User-Agent: curl/7.54.0 > Accept: */* > < HTTP/1.1 200 OK < Server: GitHub.com < Date: Tue, 10 Jul 2018 10:57:11 GMT
138. HTTP Proxy: Aleph Client • import part of global HTTP infrastructure • used pretty heavily even for internal networks (yeah, servise mesh ! ) • long story, available in Aleph since • implementaion in not compatible with clj-http API, works on the connection-pool level only • heavy lifting is done by io.netty/netty-handler-proxy
139. HTTP Proxy: Aleph Client • supports: • HTTP proxy w/o tunneling • HTTP proxy w/ tunneling (using CONNECT) • Socks4, Socks5
140. HTTP Proxy: Aleph Client • understands • auth params • proxy headers • keep-alive • proxy connection timeout
141. HTTP Proxy: Aleph Client (def proxy-pool (http/connection-pool {:connection-options {:proxy-options {:host "103.43.40.96" :port 8080 :protocol :http ;; default :tunnel? true :keep-alive? true ;; default :http-headers {"X-Via" "Proxy!"}}}})) (d/chain' (http/get "http://netty.io/" {:pool proxy-pool}) :status) ;; 200
142. HTTP Proxy: Aleph Client • when necessary, client/pipeline-builder updates Pipeline with 2 handlers • proxy handler chooses proxy implementation • pending proxy connection handler to track connection timeout • (when we're waiting on HTTP/1.1 200 Connection established) • everything else is a configuration juggling
143. TLS, ALPN
144. ALPN • Application-Layer Protocol Negotiation Extention, RFC 7301 • allows the application layer to negotiate which protocol should be performed • replaced NPN (Next Protocol Negotiation Extension) • emerged from SPDY development
145. ALPN $ curl -v https://github.com/ * Trying 192.30.253.112... * TCP_NODELAY set * Connected to github.com (192.30.253.112) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * TLSv1.2 (OUT), TLS handshake, Client hello (1): .... * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 * ALPN, server accepted to use http/1.1 * Server certificate: .... * SSL certificate verify ok. > GET / HTTP/1.1 > Host: github.com > User-Agent: curl/7.54.0 > Accept: */* > < HTTP/1.1 200 OK
146. Negotiation with Netty • SPDY server gives a good example • pass ApplicationProtocolConfig to SslContextBuilder • choose NPN or ALPN • tell acceptable protocols
147. Negotiation with Netty • handler extends ApplicationProtocolNegotiationHandler • defines fallback protocol • configures Pipeline when protocol is negotiated (appropriately)
148. Negotiation with Netty • SslHandler performs negotiation • so, it should be added to the Pipeline earlier • passes to different engines, like OpenSSL, BoringSSL or even JDK • "Don't use the JDK for ALPN! But if you absolutely have to, here's how you do it... :)", grpc-java
149. HTTP/2
150. HTTP/2 • major revision of the HTTP/1.1, SPDY successor, RFC 7540 • high-level compatibility with HTTP/1.1 • features (notably): • compressed headers (HPACK) • server push • multiplexing over a single TCP connection • more!
151. HTTP/2 • cool in theory • abnormaly complex implementation (inherently complex) • is supported by modern browsers, popularized by gRPC
152. HTTP/2 • even tho' Netty 4.1 supports HTTP/2 • Aleph does not • Ring spec does not cover all HTTP/2 features • could be done in a separate library • with smart fallback to Aleph on ALPN (when necessary) • started working, very slow progress
154. Having Fun!
155. What's Next? • [done] to cover req/resp representations • to talk about cookies & CookieStore • to talk about HTTPS • [done] talk about ALPN • [in progress] more pull requests!
156. Thanks! q&a please