Upgrade to Pro — share decks privately, control downloads, hide ads and more …

You Don't Know Node.js

You Don't Know Node.js

Node.js can be a powerful platform used in the right situations, but it also has more footguns than PHP does for new developers. Asynchronous programming style, concurrency issues, error handling, the event loop, and no default timeouts for HTTP requests or network calls will give new developers more than their fair share of troubles and can cause issues that are difficult to debug and reproduce. This talk gives a brief introduction to node.js and the event loop model (don’t block the event loop!), and covers common pitfalls when using node.js with simple and practical solutions.

vlucas

May 08, 2015
Tweet

More Decks by vlucas

Other Decks in Programming

Transcript

  1. Who are You? • Vance Lucas • http://vancelucas.com • Twitter:

    @vlucas • Co-founded JavaScript User Group • Node.js based startup postnix.com
  2. What is Node? • JavaScript on the server • V8

    engine (Google Chrome team) • Fully async non-blocking I/O • No-frills vanilla JS - not a framework
  3. Node Process • Runs forever • Does not cleanup/tear down

    • Need to watch memory usage • Callbacks for everything • Single threaded?* * or is it?
  4. Everything is async db.users.find({ name: 'John' }, function(err, users) {

    users.forEach(function(user) { console.log(user); }); });
  5. How Event Loop Works • http://latentflip.com/loupe/ • Philip Roberts: What

    the heck is the event loop anyway? • https://www.youtube.com/watch?v=8aGhZQkoFbQ
  6. Don’t Block The Event Loop! • Don’t do heavy I/O,

    large file parsing, or computation in your main process • All major processing in queue • node-blocked NPM package • https://github.com/tj/node-blocked
  7. Database • How are you connecting to your db? •

    Are you using a connection pool? • Remember: Node runs forever, and does not cleanup after each request
  8. Before var options = require('../config') // ... , mysql =

    require('mysql') , connection = mysql.createConnection(options.mysql); connection.query('SELECT id, setting_name, setting_value FROM settings', function(err, results) { // ... });
  9. After var options = require('../config') // ... , mysqlPool =

    require('./mysqlPool') // Use MySQL pool for queries var connection = mysqlPool.create({ logSql: false }, "DBPool", options.mysql); connection.query('SELECT id, setting_name, setting_value FROM settings', function(err, results) { // ... }); https://github.com/felixge/node-mysql/issues/778#issuecomment-41077643
  10. Streams • Idiomatic node • Use streams to copy file

    contents - don’t read it into memory • Gulp.js vs. Grunt.js • NPM ‘multipipe’ package • https://github.com/juliangruber/multipipe
  11. File Copy Stream var fs = require('fs'); var zlib =

    require('zlib'); fs.createReadStream('input.txt.gz') .pipe(zlib.createGunzip()) .pipe(fs.createWriteStream('output.txt'));
  12. Error Handling • Node process crashes & exits on error

    • Need to catch and handle all errors • Try/catch doesn’t work like you think • Promises can help
  13. Naive Try/Catch var fs = require('fs'); try { fs.readFile('doesnotexist.txt', function(err,

    data) { throw new Error('File does not exist!'); }); } catch(e) { console.log('There was an error, but it\'s cool, because I handled it, right?'); }
  14. Try/Catch Blocks • Callbacks will be executed in a different

    stack than your try/catch block • They are basically useless unless they are everywhere
  15. Callback Style fs.readFile("file.json", function(err, json) { if(err) { console.error("unable to

    read file"); } else { try { json = JSON.parse(json); console.log(json.success); } catch(e) { console.error("invalid json in file"); } } });
  16. Promises var Promise = require('bluebird'); var fs = Promise.promisifyAll(require("fs")); fs.readFileAsync("file.json")

    .then(JSON.parse) .then(function(json) { console.log(json.success); }) .catch(SyntaxError, function(e) { console.error("invalid json in file"); }) .catch(function(e) { console.error("unable to read file") });
  17. Promises ns.mxLookup = function(domain) { return ns.mxLookupLocal(domain) .then(function(mxRecords) { if

    (!mxRecords) { return ns.mxLookupRemote(domain); } return mxRecords; }); }; Code from postnix.com
  18. ns.checkSingleEmail = function(user, email, tag, ip) { return new Promise(function(fulfill,

    reject) { // First, see if user can check an email return ns.canUserCheckEmail(user, ip) .then(function() { // Local db lookup for cached result return ns.localEmailLookup(user, email, tag, ip); }, function(err) { // Error (no credits or restricted by IP) reject(err); }) .then(function(json) { if (json) { // Return cached JSON response immediately return fulfill(json); } // Perform all lookups return ns.mxLookup(domain) .then(function(mxRecords) { if (!mxRecords) { throw new Error('no MX records exist for domain'); } return ns.smtpCheck(user, email, mxRecords); }) .then(function(json) { fulfill(json); }); }) .catch(function(err) { if(err.code == 'ECONNREFUSED') { err.json = ns.errorJson(email, 'connection to mail server refused'); } else { err.json = ns.errorJson(email, err.message || 'connection timeout'); } db.query("DELETE FROM email_mx WHERE domain = ?", [domain]); reject(err); }); }); };
  19. Network I/O • No default timeouts for network requests •

    Sometimes you still have to .end() the connection itself on timeout event • This is on purpose, by design
  20. Adding Timeouts server.on('connection', function(socket) { log.info('SOCKET OPENED'); socket.on('end', function() {

    log.info('SOCKET END: other end of the socket sends a FIN packet'); }); // Set & listen for timeout socket.setTimeout(2000); socket.on('timeout', function() { log.info('SOCKET TIMEOUT'); socket.end(); // have to sever/end socket manually }); });
  21. ES6 Features • https://github.com/lukehoban/es6features • Use io.js! (node fork w/ES6)

    • Generators, • Workers in io.js (experimental) • https://github.com/iojs/io.js/pull/1159
  22. More Resources • Mixu’s Node Book • http://book.mixu.net/node/ • Event

    Loop Overview from Mozilla • https://developer.mozilla.org/en-US/docs/Web/ JavaScript/EventLoop