Load Balancing 101

Brief overview of loadbalancing, terminology, tools and the fact that it's really just queueing

Transcript

1. Load balancing 101 Smoke and mirrors.
2. What is load balancing Spreading load over multiple workers, servers, cores, processes, and time. • A queue of discrete requests/jobs/transactions and workers which can complete this work • A slew of concurrent, long lived connections
3. When do you have a load balancing problem? Always.
4. What can you load balance? • HTTP level load balancing • Socket level load balancing • Queue + slow legacy system
5. Queue everything and delight everyone (when you can, at the right priority)
6. Every man and his dog has a load balancer these days • HAProxy • Nginx • Amazon ELB • Varnish • Pound • Apache mod_proxy_balancer • Gearman • Celery • ØMQ • keepalived etc.
7. Load balancing terms • Hardware: ◦ Fast/Efficient ◦ Fixed circuitry/purpose ◦ In-stream decisions ◦ Reason people get Cisco qualifications ◦ SPoF (can be mitigated) • Software ◦ Plethora of options ◦ Performance tradeoffs ◦ More cleverness ◦ More deployability
8. Scheduling • Round robin • Least-connections • Random! • Clever ◦ Make decisions based on latency, worker load, geography, etc. (e.g. ELB) ◦ Dynamic: see Varnish DNS Director and Nginx Resolver / Nginx lua ◦ Hash Based: Consistent hashing based on incoming request data: see haproxy map Links: http://rapgenius.com/James-somers-herokus-ugly-secret-lyrics
9. Session stickyness / Persistence: • Sticky: route requests from known/seen requestors to the same backend worker ◦ Source IP ◦ Cookie-based ◦ How does this work in high availability? • Non-sticky: don't stick to backend workers ◦ Means you have to be clever about shared session state
10. Health checks • Do them right • Check frequently, but not too much • Check that you're getting a valid response from the backend ◦ not just that port 80 is open ◦ is it accepting http? ◦ is the page loading? ◦ is it loading correctly (ie, is the DB up?) • Build a url which gives you good health indicators, and tell the LB to check it • Spend time configuring your LB error pages
11. Example 1 - naïveté level 8 • Requests land on http://router.myapp.com • Redirect with HTTP 302 Found to http://web {1,2,3}.myapp.com Kakness: • Discuss
12. Example 2.0 - Let's use hardware! • Buy a hardware load balancer • Plug it in • Packets are free to roam! Kakness • Discuss
13. Example 2.1 - let's use software! • User hits http://myapp.com • Proxies to backend worker servers • Responses flow freely Kakness: • Discuss
14. Example 3 - let's use DNS! • User requests http://myapp.com • DNS round robins you an IP • Be careful about TTL! Kakness • Discuss
15. Cool links Varnish cleverness: http://dev.theladders.com/2013/05/varnish-in-five-acts/ Link header: http://blog.kevburnsjr.com/tagged-cache-invalidation Nginx cleverness: http://wiki.nginx.org/HttpUpstreamConsistentHash Heroku's ugly secret: http://rapgenius.com/James-somers-herokus-ugly-secret-lyrics
16. Thanks? Collaborated by • Adrian Moisey • Bradley Whittington • Bearnard Hibbins • Jonathan Hitchcock • Simon de Haan