Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Planning for the Horizontal: Scaling Node.js Ap...

Planning for the Horizontal: Scaling Node.js Applications

Slides from Brandon Cannaday's talk at HTML5 Developer Conference on how to horizontally scale Node.js applications. Covers everything from overview of architecture to specific linux settings.

Modulus

April 02, 2013
Tweet

More Decks by Modulus

Other Decks in Programming

Transcript

  1. WHEN TO SCALE? 1. RESPONSE TIMES 2. CPU USAGE 3.

    CONCURRENT CONNECTIONS Thursday, April 4, 13
  2. ULIMIT PER PROCESS FILE DESCRIPTOR LIMIT 1. Edit /etc/security/limits.conf 2.

    Add the following: * soft nofile 65535 * hard nofile 65535 root soft nofile 65535 root hard nofile 65535 Thursday, April 4, 13
  3. HURTLE: THE CPU BUY A BIGGER BOX SERVER > NODE

    SERVER > NODE 1 CORE 4 CORES Thursday, April 4, 13
  4. CLUSTER MODULE SERVER > NODE > NODE > NODE >

    NODE mydomain.com Thursday, April 4, 13
  5. CLUSTER EXAMPLE var cluster = require('cluster'); var http = require('http');

    var numCPUs = require('os').cpus().length; if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); } } else { http.createServer(function(req, res) { res.writeHead(200); res.end('Hello World.'); }).listen(80); } The Cluster Module Thursday, April 4, 13
  6. CLUSTER EXAMPLE var cluster = require('cluster'); var http = require('http');

    var numCPUs = require('os').cpus().length; if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); } } else { http.createServer(function(req, res) { res.writeHead(200); res.end('Hello World.'); }).listen(80); } Fork Children Thursday, April 4, 13
  7. CLUSTER EXAMPLE var cluster = require('cluster'); var http = require('http');

    var numCPUs = require('os').cpus().length; if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); } } else { http.createServer(function(req, res) { res.writeHead(200); res.end('Hello World.'); }).listen(80); } Handle Requests Thursday, April 4, 13
  8. ROLLING UPDATES 1. UPDATE SCRIPT 2. WORKER -> STOP LISTENING

    3. KILL WORKER 4. CALL FORK() AGAIN Thursday, April 4, 13
  9. CLUSTER MODULE SERVER > NODE > NODE > NODE >

    NODE mydomain.com Thursday, April 4, 13
  10. HURTLE: SHARED STATE SERVER > NODE > NODE > NODE

    > NODE NO SHARED STATE Thursday, April 4, 13
  11. INSTALL REDIS SERVER > NODE > NODE > NODE >

    NODE REDIS Thursday, April 4, 13
  12. EXAMPLE 1: SESSION var express = require('express'), app = express();

    app.use(express.cookieParser()); app.use(express.session({ secret: 'My Cookie Signing Secret' })); app.get('/', function(req, res) { req.session.somekey = 'some value'; }); MEMORY STORE Thursday, April 4, 13
  13. EXAMPLE 1: SESSION var express = require('express'), RedisStore = require('connect-redis')(express),

    app = express(); app.use(express.cookieParser()); app.use(express.session({ store: new RedisStore({ host: 'localhost', port: 6379 }), secret: 'My Cookie Signing Secret' })); app.get('/', function(req, res) { req.session.somekey = 'some value'; }); REDIS STORE Thursday, April 4, 13
  14. EXAMPLE 2: SOCKET.IO var RedisStore = require('socket.io/lib/stores/redis') , redis =

    require('socket.io/node_modules/redis') , pub = redis.createClient() , sub = redis.createClient() , client = redis.createClient(); io.set('store', new RedisStore({ redisPub : pub , redisSub : sub , redisClient : client })); https://github.com/LearnBoost/Socket.IO/wiki/Configuring-Socket.IO Thursday, April 4, 13
  15. RUNNING SMOOTH SERVER > NODE > NODE > NODE >

    NODE REDIS mydomain.com Thursday, April 4, 13
  16. LAST HURTLE: HORIZONTAL APP SERVER A > NODE > NODE

    > NODE > NODE REDIS APP SERVER B > NODE > NODE > NODE > NODE REDIS Thursday, April 4, 13
  17. SEPARATE REDIS APP SERVER A > NODE > NODE >

    NODE > NODE APP SERVER B > NODE > NODE > NODE > NODE REDIS SERVER Thursday, April 4, 13
  18. LOAD BALANCING APP SERVER A > NODE > NODE >

    NODE > NODE APP SERVER B > NODE > NODE > NODE > NODE REDIS SERVER LOAD BALANCER SERVER mydomain.com Thursday, April 4, 13
  19. NGINX http { upstream mydomain_com { server host1.mydomain.com:80; server host2.mydomain.com:80;

    } server { listen 80; server_name www.mydomain.com; location / { proxy_pass http://mydomain_com; } } } LOAD BALANCER SERVER Thursday, April 4, 13
  20. BOUNCY bouncy module var bouncy = require('bouncy'); var hosts =

    [ 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80); Thursday, April 4, 13
  21. BOUNCY Server collection var bouncy = require('bouncy'); var hosts =

    [ 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80); Thursday, April 4, 13
  22. BOUNCY Create server var bouncy = require('bouncy'); var hosts =

    [ 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80); Thursday, April 4, 13
  23. BOUNCY Bounce request var bouncy = require('bouncy'); var hosts =

    [ 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80); Thursday, April 4, 13
  24. NGINX AFFINITY http { upstream mydomain_com { sticky; server host1.mydomain.com:80;

    server host2.mydomain.com:80; } server { listen 80; server_name www.mydomain.com; location / { proxy_pass http://mydomain_com; } } } Thursday, April 4, 13
  25. RUNNING SMOOTH APP SERVER A > NODE > NODE >

    NODE > NODE APP SERVER B > NODE > NODE > NODE > NODE REDIS SERVER LOAD BALANCER SERVER mydomain.com Thursday, April 4, 13
  26. ROLLING UPDATES 1. REMOVE APP SERVER FROM LOAD BALANCER 2.

    UPGRADE APP SERVER 3. ADD BACK 4. REPEAT Thursday, April 4, 13
  27. SSL APP SERVER A > NODE > NODE > NODE

    > NODE APP SERVER B > NODE > NODE > NODE > NODE REDIS SERVER LB SERVER SSL SSL TERMINATOR Thursday, April 4, 13
  28. STUD https://github.com/bumptech/stud frontend = [*]:443 backend = [127.0.0.1]:80 ssl =

    on pem-file = "myCert.pem" EXAMPLE CONFIG FILE Thursday, April 4, 13
  29. RUNNING SMOOTH W/SSL APP SERVER A > NODE > NODE

    > NODE > NODE APP SERVER B > NODE > NODE > NODE > NODE REDIS SERVER LB SERVER SSL mydomain.com Thursday, April 4, 13
  30. ROUND-ROBIN DNS CLIENT 1 1. xxx.xxx.xxx.x 2. xxx.xxx.xxx.y CLIENT 2

    1. xxx.xxx.xxx.y 2. xxx.xxx.xxx.x Thursday, April 4, 13