Accelerating Web Applications with Varnish

The Varnish web accelerator has been the center of attention recently as more and more content providers realize its power and flexibility. More than just a simple caching proxy, Varnish lets us crack open the request - response cycle and take control of precisely how each and every request is served. Through practical examples, I'll demonstrate how websites of every size are using this powerful tool to maximize performance, improving the user's experience while eliminating back-end complexity. We'll explore the unique Varnish Control Language and discover how it can make your site delivery lighter, leaner, and lightning fast!

Transcript

1. ACCELERATING WEB APPLICATIONS WITH VARNISH SAMANTHA QUIÑONES, 20 MAY 2014, PHP TEK
2. About Me @ieatkillerbees http://tembies.com Decade+ as a SW engineer at Visa International Lead PHP developer at POLITICO since March, 2013 Software engineer since 1996 working primarily with Python and PHP since 2005
3. In the beginning… The web was born as a platform for mostly static content Most content, once published, changes rarely Static content continues to dominate the web
4. What is static? Even though most content is static, changes must be available very rapidly Static content is cheap to deliver but expensive to maintain Static content is... static. Stale. Boring.
6. What is dynamic? Content is curated for each consumer Dynamic content is expensive to deliver but cheap to maintain. Dynamic content makes the web engaging.
7. An age-old question Computational vs Human resources Static vs Dynamic
9. “The computer is incredibly fast, accurate, and stupid. Man is unbelievably slow, inaccurate, and brilliant. The marriage of the two is a force beyond calculation.” –Leo Cherne
10. UNDERSTANDING CONTENT DELIVERY
11. Basic content-delivery via HTTP Files on disk Passive benefit from OS memory mapping and caching Prone to I/O blocking Nearly impossible to scale out
12. The request-response cycle
13. Dynamic Backends
14. The Relational DBMS High degree of data organization and integrity Common, well-supported interfaces SQL Designed to scale on mainframes and “enterprise” servers (read: more CPUs/engines, more RAM/real storage)
15. The Relational DMBS Data normalization Scales up, not out
16. The Relational DBMS SELECT DISTINCT t1.CU_ship_name1, t1.CU_ship_name2, t1.CU_email FROM customers AS t1 WHERE t1.CU_solicit=1 AND t1.CU_cdate >= 20100725000000 AND t1.CU_cdate <= 20100801000000 AND EXISTS( SELECT NULL FROM orders AS t2 INNER JOIN item AS t3 ON t2.O_ref = t3.I_oref INNER JOIN product AS t4 ON t3.I_pid = t4.P_id INNER JOIN ( SELECT C_id FROM category WHERE C_store_type = 2 ) AS t5 ON t4.P_cat = t5.C_id WHERE t1.CU_id = t2.O_cid);
17. How do we leverage the power of a DBMS to manage content while also building applications that are scalable, performant, and durable?
19. “The advantage of adding cache…is that they have the potential to… eliminate some interactions, improving efficiency, scalability, and user-perceived performance by reducing the average latency of a series of interactions.” –Roy Fielding
20. Caches Amortize the cost of expensive operations across many requests Vastly increase the performance of an application at the cost of responsiveness to change Caches can scale horizontally in front of a smaller DBMS implementation
21. Caches of all Flavors In-memory object caches Content-delivery networks (CDNs) Static pre-rendering (yeah, this happens) Key-value stores & Document Databases Caching proxies
22. Caches of all Flavors Memcached is among the most popular caching tools In-memory key-value store Operates on the predominant caching pattern
23. In-Memory Object Caches $cache = new CacheClient(['127.0.0.1:4242']); ! $widget = $cache->get('widget-1'); if (!$widget) { $widget = $database->find('widget-1'); $cache->set('widget-1', $widget, CACHE_EXPIRY); } ! $gizmo = new Gizmo(); $id = $database->save($gizmo); $cache->set($id, $gizmo, CACHE_EXPIRY);
24. In-Memory Object Caches Tools like memcached are developer-focused The onus is on developers to make sure that they are caching data efficiently Usually this means that absolutely everything gets cached Or, the caching platform is underutilized to the point of near- irrelevance.
25. Content Delivery Networks Akamai, Cloudflare, etc Fantastic for caching truly static content Extremely robust Largely invisible to developers Not at all flexible
26. Static Pre-Rendering Very annoying to maintain When combined with other methods, it can greatly reduce the “freshness” of content Hides extremely poor performance Extremely robust
27. Caching Reverse Proxies Almost like a “local” CDN Maintain a cache of content from the origin server(s) Invisible to developers Squid, Varnish, and other “web accelerators” fall in this category
28. All roads lead… We're trying to make our sites “faster”… But what does that mean? Most dynamic content is pretty static
30. THIS TALK COVERS MOSTLY VARNISH 3 NOTA BENE
31. The reality of media publishing Content is overwhelmingly static Yet extremely time-sensitive Slow delivery of content is a huge revenue risk Load is inconsistent and unpredictable Editorial and engineering requirements are rarely in sync Failure can be devastating
32. Case Study: A Norwegian Tabloid Verdens Gang is one of Norway's most popular newspapers Suffered the same problems of all media platforms Poul-Henning Kamp, a BSD core developer, was the lead developer and application architect for Verdens Gang As a kernel developer, Kamp has a particular set of skills that allowed him to approach this problem from a new angle
35. Memory & Storage In the olden days, there was a line of demarcation between primary and secondary storage In short, primary storage (RAM in modern computers) can be accessed directly by the CPU Secondary storage is accessed via an I/O channel or controller
36. Memory and Storage As early as the 1950s, computer scientists were experimenting with virtual memory. By the 1970s, virtual memory was common in commercial computers Virtual memory is an abstraction that allows secondary storage to extend primary storage The operating system cooperates with specialized hardware to manage the paging of data in and out of virtual memory.
37. Virtual Memory is a Cache In essence, virtual memory is a cache The operating system swaps data between high-speed primary storage and slower secondary storage based on factors like age and access frequency Commonly accessed data is kept warm and ready while rarely- needed data can be quickly retrieved when called for
38. Caching-Reverse Proxies Traditional caching reverse proxies allocate memory and fill it with objects Less-used objects are written to disk. Objects on disk are written to memory when requested Sounds familiar, right?
39. Varnish is different The operating system is already paging data between primary and secondary storage So why reinvent the wheel? Varnish gets out of the operating system's way and lets the kernel do what it's best at!
40. Varnish in a nutshell At start-up, Varnish allocates a big, empty chunk of memory Within that space, Varnish maintains a workspace with pointers to cached objects, headers, etc Varnish prioritizes worker threads by most recently used These factors combine to reduce overall memory ops
42. Getting Started Open-source: http://www.varnish-cache.org Commercial: https://www.varnish-software.com/ DEB: apt-get install varnish RPM: Available from EPEL Open BSD: pkg_add varnish Source: git://git.varnish-cache.org/varnish-cache
43. Varnish Config Language Varnish has its own DSL, VCL Modeled after C (and allows for inline C) VCL allows us to modify requests and responses “in-flight” Configure “backends” and how to interact with them Compiled in to binary objects and loaded in to memory
44. Starting varnishd -f : Specify a VCL file to load -s : Configure the storage (malloc, file, etc) -T : Specify where Varnish should listen for admin connections -a : Specify where Varnish should listen for HTTP requests
45. Backends Backends are origin servers (Apache, NGINX, etc) that will be serving your content Varnish can proxy a single backend, a cluster of them, or multiple clusters of them The configuration of backends and request routing could fill an hour on its own
46. Backends backend default { .host = "127.0.0.1"; .port = "80"; }
47. Directors Logical clusters of backends that allow for load balancing and redundancy Monitor the health of individual backends Random directors route requests to origin servers randomly Client directors route based on the identity of the client Hash directors route based on the URL of the request Round-robin directors round robin requests
48. DNS Directors director originpool dns { .list = { .host_header = "origin.foo.com"; .port = "80"; .connect_timeout = 0.5s; "10.42.42.0"/24; } .ttl = 5m; }
49. Probes Used by Varnish determines if a backend is healthy or not Specifies a check interval, timeout, expected response, etc. Can be set as part of a backend definition, or standalone.
50. Probes probe healthcheck { .url = "/health.php"; .interval = 60s; .timeout = 0.3 s; .window = 8; .threshold = 3; .initial = 3; .expected_response = 200; } backend origin { .host = "origin.foo.com"; .port = "http"; .probe = healthcheck; }
51. ACLs Used to identify client addresses Can allow bypassing the proxy for local clients Restrict URLs to certain clients
52. ACLs acl admin { "localhost"; "10.42.42.42"; } sub vcl_recv { if (req.url ~ "^/admin") { if (client.ip ~ admin) { return(pass); } else { error 405 "Not allowed in admin area."; } } }
53. Hooks Hooks allow the execution of VCL at a number of pre-defined points in the request-response cycle. vcl_recv – Called after a request has been received vcl_pipe – Called when entering pipe mode vcl_pass – Called when entering pass mode vcl_hit – Called when an object is found in the cache vcl_miss – Called when an object is not found in the cache
54. Hooks vcl_fetch* – Called when a response has been received from an origin server vcl_deliver – Called before a cached object is returned to a client vcl_error – Called when an error happens
55. Hooks in Varnish 4 vcl_fetch has been replaced by: vcl_backend_fetch - Called before sending the backend request vcl_backend_response - Called after a response has been successfully retrieved from the backend
56. Grace Mode Allows varnish to serve an expired object while a fresh object is being generated by the backend. Doesn't require the user to pay a “first-hit” tax Avoids threads piling up Protects against stampeding popular resources Some users may get expired data even though fresh data is “technically” available.
57. Saint Mode Like grace mode, but more awesome. Allows Varnish to serve expired objects when we don't like what the backends are returning. 200 OK with no response body? Serve from cache 500 errors? Serve from cache Merged in to Grace Mode in Varnish 4
58. Saint Mode Receive request for expired resource Request resource from origin-1 Receive 503 Request resource from origin-2 Receive 503 Increase TTL for resource by 30 seconds and restart request
59. Managing Varnish Varnish has a simple command prompt accessible by telnetting to the configured port List loaded VCL files, load new VCL files, switch between active VCL files Get the status of backends“ban” URLs (force them to pass requests to a backend)
60. Purging Objects Varnish understands a nonstandard HTTP method “PURGE” PURGE /obsolete/resource Configurable via ACL Frees memory Next client request will refresh the content
61. Purging Objects sub vcl_recv { if (req.request == "PURGE") { if (!client.ip ~ purge) { error 405 "Not allowed."; } return (lookup); } } sub vcl_hit { if (req.request == "PURGE") { purge; error 200 "Purged."; } } sub vcl_miss { if (req.request == "PURGE") { purge; error 200 "Purged."; } }
62. Banning Objects Banning applies a filter to cached objects Another way to invalidate content Applies instantly Does NOT free up memory Large ban lists can degrade performance Understands regular expressions
63. Vary Special header used by the origin server to indicate one resource is a variation of another Variations are common for different encodings (gzip, deflate) Or based on user agent (but it’s important to normalize!)
64. Vary if (req.http.Accept-Encoding) { if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") { # No point in compressing these remove req.http.Accept-Encoding; } elsif (req.http.Accept-Encoding ~ "gzip") { set req.http.Accept-Encoding = "gzip"; } elsif (req.http.Accept-Encoding ~ "deflate") { set req.http.Accept-Encoding = "deflate"; } else { # unknown algorithm remove req.http.Accept-Encoding; } }
65. Edge-Side Includes Varnish supports a subset of ESI esi:include, esi:remove, <!—esi … —>
66. Replaying Traffic Takes a Varnish log and replays all of the traffic Powerful tool to quickly warm up an empty cache when starting new Varnish instances
67. Varnish is complex Varnish is an incredibly complex and powerful tool Varnish is a dynamic caching framework This has been an overview of its features.Install it, read the docs, play, experiment, and explore!
69. Varnish killed the memcached star Politico's PHP stack uses a distributed service-oriented architecture Nearly all operations are abstracted behind RESTful APIs REST services from MongoDB, MySQL, other internal and external APIs We don't use memcached We don't use anything LIKE memcached
70. Varnish killed the memcached star Our services network sits behind a cluster of Varnish instances. All calls, even internally, go through those Varnish clusters. Every service returns appropriate HTTP status codes, including a 502s, 503s, and 504s. Every service returns Cache-Control headers which instruct Varnish to set the TTLs for each resource
71. We consume a lot of stuff Stored content from our internal CMS Static assets in S3 Data feeds from other reporting agencies Every external dependency is abstracted behind an interface that we define and control
72. Our Request-Response Cycle
73. What to do when an external dependency fails? Option 1: Fail gracefully Option 2: Fail ungracefully Option 3: Don't fail, because you're a BEAST!
74. Invisible fault tolerance
75. Summary Varnish is powerful. It's simple to get started, and difficult to master Varnish has the potential to not only accelerate applications, but to simplify infrastructures
76. Varnish 4 Addendum VCL “Objects” Modularized backends & directors
77. Contact Info Twitter: @ieatkillerbees Blog: http://tembies.com Email: samantha@tembies.com Joind.in: https://joind.in/10648
78. Contact Info & Feedback Twitter: @ieatkillerbees Blog: http://tembies.com Email: samantha@tembies.com Joind.in: https://joind.in/10648