Faster webapp: understand the network

Many web developers treat the browsers as a black box and ignore all the complexity of the network. This talk is an introduction to the excellent book "High Performance Browser Networking" by Ilya Grigorik .

Transcript

1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Faster webapp Grégoire Charvet greg@geekingfrog.com April 30, 2014 https://speakerdeck.com/geekingfrog/faster-webapp-understand-the-network Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 1 / 65
2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Overview 1 Networking 101 UDP TCP TLS 2 HTTP 3 Final thoughts Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 2 / 65
3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Disclaimer Go read this book −→ Everything and much more is in here. Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 3 / 65
4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Intro Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 4 / 65
5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Speed is a feature Faster sites lead to: better user retention better user engagement higher conversion Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 5 / 65
6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Two critical components Figure: Speed and Latency Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 6 / 65
7. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Route Distance Time, light in a vac- uum Time, light in a fiber Taipei to Tokyo 2115 km 7 ms 10 ms Taipei to Paris 9845 km 32 ms 49 ms Taipei to SF 10346 km 34 ms 51 ms Equatorial cir- cumference 40 075 km 133 ms 200 ms Table: Some latencies Rule of thumb: speed of light in fiber: 2.108m/s Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 7 / 65
8. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Measure latency traceroute (linux) tracert (windows) $> traceroute google.com traceroute to google.com (74.125.224.102), 64 hops max, 52 byte packets 1 10.1.10.1 (10.1.10.1) 7.120 ms 8.925 ms 1.199 ms 2 96.157.100.1 (96.157.100.1) 20.894 ms 32.138 ms 28.928 ms 3 x.santaclara.xxxx.com (68.85.191.29) 9.953 ms 11.359 ms 9.686 ms 4 x.oakland.xxx.com (68.86.143.98) 24.013 ms 21.423 ms 19.594 ms 5 68.86.91.205 (68.86.91.205) 16.578 ms 71.938 ms 36.496 ms 6 x.sanjose.ca.xxx.com (68.86.85.78) 17.135 ms 17.978 ms 22.870 ms 7 x.529bryant.xxx.com (68.86.87.142) 25.568 ms 22.865 ms 23.392 ms 8 66.208.228.226 (66.208.228.226) 40.582 ms 16.058 ms 15.629 ms 9 72.14.232.136 (72.14.232.136) 20.149 ms 20.210 ms 18.020 ms 10 64.233.174.109 (64.233.174.109) 63.946 ms 18.995 ms 18.150 ms 11 x.1e100.net (74.125.224.102) 18.467 ms 17.839 ms 17.958 ms Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 8 / 65
9. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Two barriers 300ms: ”There is some lag” 1s: feels slugish ← upper limit to aim for. Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 9 / 65
10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Latency matter more than bandwidth Figure: Bandwidth vs latency Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 10 / 65
11. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . UDP User datagram protocol Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 11 / 65
12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simplest protocol - UDP Super simple Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 12 / 65
13. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A few shortcomings No state No acknowledgement No congestion control No message splitting No order delivery garantee Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 13 / 65
14. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TCP Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 14 / 65
15. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TCP (transfer control protocol) Address the issues of UDP Building block of the web Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 15 / 65
16. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Three-Way handshake Figure: Three-Way handshake Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 16 / 65
17. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Flow control receive window (rwnd) to control the size of buffers Slow start & Congestion avoidance Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 17 / 65
18. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Receive window Figure: Receive window advertisement Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 18 / 65
19. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Slow start Avoid congestion collapse by starting sending few packets Measure available bandwidth by sending packet Dropped packet (missed ACK) trigger adjustements of cwnd Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 19 / 65
20. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Figure: Packet dropped Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 20 / 65
21. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Slow start and congestion avoidance Automatic control of the congestion window Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 21 / 65
22. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Time required to reach maximum throughput Time to reach a cwnd of size N; Time = RTT × ⌈ log 2 ( N init cwnd )⌉ Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 22 / 65
23. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example Client and server rwnd: 65535 bytes (64K) Initial congestion window: 4 segments (RFC 2581) Roundtrip time: 56 ms (London to New York) Size of one segment: 1460 bytes (MTU - ipv4 overhead) Number of segments required to get to 64K: 65 535 bytes 1460 bytes ≈ 45 segments Time for cwnd to reach 45 segments: 56 ms × ⌈ log 2 ( 45 4 )⌉ = 224 ms Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 23 / 65
24. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example cont’d Figure: Slow start and congestion window growth Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 24 / 65
25. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implications FOUR round trip ! Bandwidth is irrelevant here Big problem for most HTTP connection (bursty traffic) Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 25 / 65
26. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fetching a file over a new TCP connection Fetch a file of 40 KB from New-York to London. Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 26 / 65
27. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Reuse of tcp connection Getting the same file over the same connection. Figure: Same file over the same connection Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 27 / 65
28. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary of TCP Three-Way handshake → full RTT of latency Slow start applied for every new connections Flow and congestion control regulate throughput Congestion window size control throughput Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 28 / 65
29. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TCP optimizations TCP fastopen: send data with the first syn packet (linux kernel ≥ 3.7) Window scaling: allow greater receive window disable slow start restart $> sysctl net.ipv4.tcp_slow_start_after_idle $> sysctl -w net.ipv4.tcp_slow_start_after_idle=0 Increase initial congestion window size to 10 (kernel ≥ 2.6.39) Reuse tcp connection Put server close to clients Compress transferred data Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 29 / 65
30. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TLS Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 30 / 65
31. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TLS or SSL ? Original name: SSL, developed at Netscape. Renamed to TLS when standardized by the IETF. Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 31 / 65
32. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Encryption, Authentication, Integrity TLS provides: Encryption Authentication Integrity As a side effect: protects whatever application is running on top of it from misconfigured middle agent (proxies). Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 32 / 65
33. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Computational cost “SSL/TLS is not computationally expensive anymore.” — Adam Langley (Google) - 2010 We have found that modern software-based TLS implementations running on commodity CPUs are fast enough to handle heavy HTTPS traffic load without needing to resort to dedicated cryptographic hardware. We serve all of our HTTPS traffic using software running on commodity hardware. — Doug Beaver (Facebook) Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 33 / 65
34. TLS handshake Figure: TLS handshake . . . .... .... .... . . . . .
35. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Optimizing TLS TLS session resumption session ticket TLS false start Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 35 / 65
36. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Session resumption Re-use an already negotiated key. Avoid a full round-trip. A must have in most circumstances. Problem on the servers: must maintain a cache for each session ID. Even more tricky when it comes to multiple servers sharing the same cache. Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 36 / 65
37. Session resumption cont’d Figure: Abbreviated handshake . . . .... .... .... . . . . .
38. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Session ticket Encrypt all session’s data and store it on the client. Only the server knows the key. No server side caching. But servers must be initialized with the same secret key (multi server behind load balancer concerns). Also need structure to deploy new keys Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 38 / 65
39. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . False start What about new visitors ? Client can send data as soon at it knows the session key. Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 39 / 65
40. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . False start Figure: TLS false start Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 40 / 65
41. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Compatibility of false start There are some issue (refer to the book for details). Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 41 / 65
42. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TLS record Figure: TLS record Record can be split in multiple TCP packets. Record must be complete to decrypt the data. Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 42 / 65
43. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Optimizing TLS record size IP overhead: 20 bytes (v4) or 40 bytes (v6) TCP overhead: 20 bytes TCP options overhead: 40 bytes usual MTU: 1500 bytes. Initial size, when to upgrade to what size ? Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 43 / 65
44. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Google’s implementation As early 2014: TLS record set to fit into one TCP packet for the first MB, then upgraded to 16KB. Reset to initial value after ≈ 1s of inactivity. Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 44 / 65
45. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TLS compression Don’t do it. CRIME attack will compress everything (including image and other already compressed stuff) Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 45 / 65
46. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HTTP Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 46 / 65
47. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rapid history HTTP 0.9 Limited to html and ascii. Connection closed after response. HTTP 1.0 Introduced status code, request and response headers. Response is not limited to html. Connection is closed after response. HTTP 1.1 Connection re-use by default Complex, tons of options (content-encoding, caching directive, language negotiation and so on...) HTTP 2.0 (spdy) Work in progress. SPDY is a working implementation. Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 47 / 65
48. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Web application requests Figure: One http request Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 48 / 65
49. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Browser optimizations Modern browsers do a lot of work under the hood to provide better experience. DNS pre-resolve and TCP pre-connect. Speculative optimization. Resources pre-fetching and prioritization. Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 49 / 65
50. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Helping the browsers <link rel=”dns-prefetch” href=”//hostname_to_resolve.com”> 1 <link rel=”subresource” href=”/javascript/myapp.js”> 2 <link rel=”prefetch” href=”/images/big.jpeg”> 3 <link rel=”prerender” href=”//example.org/next_page.html”> 4 1: pre-resolve hostname. 2: pre-fetch critical resource used later in the page. 3: pre-fetch resource for future navigation. 4: pre-render page for future navigation. Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 50 / 65
51. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Browser support Browser dns-prefetch subresource prefetch prerender Firefox 3.5+ n/a 3.5+ n/a Chrome 1.0+ 1.0+ 1.0+ 13+ Safari 5.01+ n/a n/a n/a IE 9+ (prefetch) n/a 10+ 11+ Table: Speculative optimization support Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 51 / 65
52. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HTTP optimizations HTTP 0.9 or 1.0 → upgrade to 1.1 Reduce DNS lookup Fewer HTTP requests Use a CDN Use HTTP caching mechanisms Gzip assets Avoid HTTP redirect (even worse if it redirect to a different host) Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 52 / 65
53. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Figure: Benefit of connection keep-alive Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 53 / 65
54. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HTTP pipelining Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 54 / 65
55. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pipelining concerns Clients and server must support it. Application must handle aborted connections. Must protect from broken intermediaries (use HTTPS) Head of line blocking The head of line blocking problem actually makes pipelining difficult to use in practice. Browsers open 6 TCP connections to a server and process requests in parallel. Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 55 / 65
56. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Multiple TCP connections In practice, browsers open 6 TCP connections per hosts. Why 6? Experimentation. No head of line blocking More bytes can be sent within the first cwnd But More complexity to manage the sockets collections More memory and cpu Limited application parallelism Resource exhaustion Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 56 / 65
57. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Figure: Resource exhaustion Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 57 / 65
58. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Domain sharding Put resources on different domain (pic1.example.com, pic2.example.com) Cost: new dns request, new tcp connections (slow start) Measure it (consider sharding when you have more than 10 resources download simultanously) Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 58 / 65
59. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HTTP optimizations, application level Concatenation & spritting (less requests) resource inlining (+33% size for images) <img src=”data:image/gif;base64,R0lGODlhAQAB AIAAAAAAAAAAACH5BAAAAAAALAAAAAABA AEAAAICTAEAOw==” alt=”1x1 transparent (GIF) pixel” /> Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 59 / 65
60. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Final thoughts Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 60 / 65
61. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Optimization checklist Always do: Reduce dns lookups. Reuse tcp connections. Minimize number of HTTP request. Use a cdn. Eliminate unecessary resources and transferred bytes. Compress data during transfer. Cache resources on the client. Parallelize resources fetching. Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 61 / 65
62. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HTTP 1.1 optimizations HTTP pipelining (if you control both client and server this is viable). Domain sharding (warnings apply). Bundle resources (less requests). Inline small resources. Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 62 / 65
63. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What’s coming next ? HTTP2 and SPDY WebRTC Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 63 / 65
64. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . And remember: measure then tweak Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 64 / 65
65. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The end Grégoire Charvetgreg@geekingfrog.com Understand the network April 30, 2014 65 / 65