Planning for the Horizontal: Scaling Node.js Applications

Slides from Brandon Cannaday's talk at HTML5 Developer Conference on how to horizontally scale Node.js applications. Covers everything from overview of architecture to specific linux settings.

Transcript

1. TheReddest brandon@modulus.io PLANNING FOR THE HORIZONTAL SCALING NODE.JS APPLICATIONS Brandon Cannaday Thursday, April 4, 13
2. ME HOSTING, DATA, STATS FOR NODE.JS modulus.io Thursday, April 4, 13
3. WHEN TO SCALE? 1. RESPONSE TIMES 2. CPU USAGE 3. CONCURRENT CONNECTIONS Thursday, April 4, 13
4. NODEFLY.COM Thursday, April 4, 13
5. STARTING POINT mydomain.com SERVER > NODE Thursday, April 4, 13
6. NODE TWEAKS http.globalAgent.maxSockets = Number.MAX_VALUE; CONCURRENT OUTGOING CONNECTION LIMIT Thursday, April 4, 13
7. HURTLE: THE SERVER LINUX CONFIGURATION 1. FILE-MAX 2. SOMAXCONN 3. ULIMIT Thursday, April 4, 13
8. FILE-MAX SYSTEM FILE DESCRIPTOR LIMIT 1. Run sysctl -w fs.file-max=65535 2. Run sysctl -p Thursday, April 4, 13
9. SOMAXCONN SOCKET LISTEN QUEUE LENGTH 1. Run sysctl -w net.core.somaxconn=65535 2. Run sysctl -p Thursday, April 4, 13
10. ULIMIT PER PROCESS FILE DESCRIPTOR LIMIT 1. Edit /etc/security/limits.conf 2. Add the following: * soft nofile 65535 * hard nofile 65535 root soft nofile 65535 root hard nofile 65535 Thursday, April 4, 13
11. RUNNING SMOOTH mydomain.com SERVER > NODE Thursday, April 4, 13
12. HURTLE: THE CPU BUY A BIGGER BOX SERVER > NODE SERVER > NODE 1 CORE 4 CORES Thursday, April 4, 13
13. MULTICORE NODE 1 2 3 4 100% CORE USAGE Thursday, April 4, 13
14. CLUSTER MODULE SERVER > NODE > NODE > NODE > NODE mydomain.com Thursday, April 4, 13
15. CLUSTER EXAMPLE var cluster = require('cluster'); var http = require('http'); var numCPUs = require('os').cpus().length; if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); } } else { http.createServer(function(req, res) { res.writeHead(200); res.end('Hello World.'); }).listen(80); } The Cluster Module Thursday, April 4, 13
16. CLUSTER EXAMPLE var cluster = require('cluster'); var http = require('http'); var numCPUs = require('os').cpus().length; if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); } } else { http.createServer(function(req, res) { res.writeHead(200); res.end('Hello World.'); }).listen(80); } Fork Children Thursday, April 4, 13
17. CLUSTER EXAMPLE var cluster = require('cluster'); var http = require('http'); var numCPUs = require('os').cpus().length; if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); } } else { http.createServer(function(req, res) { res.writeHead(200); res.end('Hello World.'); }).listen(80); } Handle Requests Thursday, April 4, 13
18. CLUSTER LISTEN WORKER listen(...) MASTER Handle Thursday, April 4, 13
19. ROLLING UPDATES 1. UPDATE SCRIPT 2. WORKER -> STOP LISTENING 3. KILL WORKER 4. CALL FORK() AGAIN Thursday, April 4, 13
20. CLUSTER MODULE SERVER > NODE > NODE > NODE > NODE mydomain.com Thursday, April 4, 13
21. HURTLE: SHARED STATE SERVER > NODE > NODE > NODE > NODE NO SHARED STATE Thursday, April 4, 13
22. INSTALL REDIS SERVER > NODE > NODE > NODE > NODE REDIS Thursday, April 4, 13
23. EXAMPLE 1: SESSION var express = require('express'), app = express(); app.use(express.cookieParser()); app.use(express.session({ secret: 'My Cookie Signing Secret' })); app.get('/', function(req, res) { req.session.somekey = 'some value'; }); MEMORY STORE Thursday, April 4, 13
24. EXAMPLE 1: SESSION var express = require('express'), RedisStore = require('connect-redis')(express), app = express(); app.use(express.cookieParser()); app.use(express.session({ store: new RedisStore({ host: 'localhost', port: 6379 }), secret: 'My Cookie Signing Secret' })); app.get('/', function(req, res) { req.session.somekey = 'some value'; }); REDIS STORE Thursday, April 4, 13
25. EXAMPLE 2: SOCKET.IO var RedisStore = require('socket.io/lib/stores/redis') , redis = require('socket.io/node_modules/redis') , pub = redis.createClient() , sub = redis.createClient() , client = redis.createClient(); io.set('store', new RedisStore({ redisPub : pub , redisSub : sub , redisClient : client })); https://github.com/LearnBoost/Socket.IO/wiki/Configuring-Socket.IO Thursday, April 4, 13
26. RUNNING SMOOTH SERVER > NODE > NODE > NODE > NODE REDIS mydomain.com Thursday, April 4, 13
27. LAST HURTLE: HORIZONTAL APP SERVER A > NODE > NODE > NODE > NODE REDIS APP SERVER B > NODE > NODE > NODE > NODE REDIS Thursday, April 4, 13
28. SEPARATE REDIS APP SERVER A > NODE > NODE > NODE > NODE APP SERVER B > NODE > NODE > NODE > NODE REDIS SERVER Thursday, April 4, 13
29. LOAD BALANCING APP SERVER A > NODE > NODE > NODE > NODE APP SERVER B > NODE > NODE > NODE > NODE REDIS SERVER LOAD BALANCER SERVER mydomain.com Thursday, April 4, 13
30. LOAD BALANCING 1. MANAGED 2. INSTALL ONE 3. WRITE YOUR OWN Thursday, April 4, 13
31. NGINX http { upstream mydomain_com { server host1.mydomain.com:80; server host2.mydomain.com:80; } server { listen 80; server_name www.mydomain.com; location / { proxy_pass http://mydomain_com; } } } LOAD BALANCER SERVER Thursday, April 4, 13
32. WRITE ONE https://github.com/substack/bouncy Thursday, April 4, 13
33. BOUNCY bouncy module var bouncy = require('bouncy'); var hosts = [ 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80); Thursday, April 4, 13
34. BOUNCY Server collection var bouncy = require('bouncy'); var hosts = [ 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80); Thursday, April 4, 13
35. BOUNCY Create server var bouncy = require('bouncy'); var hosts = [ 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80); Thursday, April 4, 13
36. BOUNCY Bounce request var bouncy = require('bouncy'); var hosts = [ 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80); Thursday, April 4, 13
37. AFFINITY SESSION AFFINITY STICKY SESSIONS SEND THE SAME PERSON BACK TO THE SAME SERVER Thursday, April 4, 13
38. NGINX AFFINITY http { upstream mydomain_com { sticky; server host1.mydomain.com:80; server host2.mydomain.com:80; } server { listen 80; server_name www.mydomain.com; location / { proxy_pass http://mydomain_com; } } } Thursday, April 4, 13
39. CUSTOM AFFINITY req.headers['x-forwarded-for'] req.connection.remoteAddress Thursday, April 4, 13
40. RUNNING SMOOTH APP SERVER A > NODE > NODE > NODE > NODE APP SERVER B > NODE > NODE > NODE > NODE REDIS SERVER LOAD BALANCER SERVER mydomain.com Thursday, April 4, 13
41. ROLLING UPDATES 1. REMOVE APP SERVER FROM LOAD BALANCER 2. UPGRADE APP SERVER 3. ADD BACK 4. REPEAT Thursday, April 4, 13
42. SSL TERMINATE EARLY Thursday, April 4, 13
43. SSL APP SERVER A > NODE > NODE > NODE > NODE APP SERVER B > NODE > NODE > NODE > NODE REDIS SERVER LB SERVER SSL SSL TERMINATOR Thursday, April 4, 13
44. SSL LB SERVER SSL mydomain.com 80 443 Thursday, April 4, 13
45. STUD https://github.com/bumptech/stud frontend = [*]:443 backend = [127.0.0.1]:80 ssl = on pem-file = "myCert.pem" EXAMPLE CONFIG FILE Thursday, April 4, 13
46. RUNNING SMOOTH W/SSL APP SERVER A > NODE > NODE > NODE > NODE APP SERVER B > NODE > NODE > NODE > NODE REDIS SERVER LB SERVER SSL mydomain.com Thursday, April 4, 13
47. HUGE REDIS SERVER LB SERVER SSL LB SERVER SSL Thursday, April 4, 13
48. DNS ROUND-ROBIN DNS MULTIPLE RECORDS, ONE DOMAIN Thursday, April 4, 13
49. ROUND-ROBIN DNS CLIENT 1 1. xxx.xxx.xxx.x 2. xxx.xxx.xxx.y CLIENT 2 1. xxx.xxx.xxx.y 2. xxx.xxx.xxx.x Thursday, April 4, 13
50. RUNNING SMOOTH REDIS SERVER LB SERVER SSL LB SERVER SSL Thursday, April 4, 13
51. BIG ENOUGH SERVER > NODE Thursday, April 4, 13
52. BIG ENOUGH SERVER > NODE Thursday, April 4, 13