Node.js London Meetup, May 2016 - Why load test, do's and don'ts of load-testing, and why Artillery.io is a great choice for load-testing, plus a quick overview of flamegraphs and heapdumps in Node.js
problems • Users find them • Or: • You crash your app in development • Or it crashes under load in production • Or: • You don’t need to worry if you like downtime (and getting paged) • Of if your users love every extra 100ms of response time
a software system or computing device and measuring its response. Load testing is performed to determine a system's behavior under both normal and anticipated peak load conditions. -Wikipedia
• Learn about the limits of your code, your dependencies, your whole stack • Example: New Relic agent for Node can reduce the performance of your up by as much as 20% under high load scenarios
• Learn about the limits of your code, your dependencies, your whole stack • Example: New Relic agent for Node can reduce the performance of your up by as much as 20% under high load scenarios • Design aid • PDD: http://bit.ly/1rndEiw or https://blog.yld.io
• Basically, a tool that can send a lot of requests to a server, very quickly • Many options (30+ listed on Wikipedia) • Where they differ is: • protocols they support • how flexible they are in allowing you to model the load • UI, UX, integrations
the box), extensible ”engine” interface. HTTP/2 and MQTT support coming. • Has a nice CLI. Integrates well with CI servers. • Very flexible for modeling realistic load - user scenarios, arrival rates, loops
the box), extensible ”engine” interface. HTTP/2 and MQTT support coming. • Has a nice CLI. Integrates well with CI servers. • Very flexible for modeling realistic load - user scenarios, arrival rates, loops • Virtual user behavior is scriptable with JS
the box), extensible ”engine” interface. HTTP/2 and MQTT support coming. • Has a nice CLI. Integrates well with CI servers. • Very flexible for modeling realistic load - user scenarios, arrival rates, loops • Virtual user behavior is scriptable with JS • Good performance (multicore)
service • Phases control arrivals • Each arrival is a new TCP socket (just like in the real world), picks a scenario, runs through it to completion or failure
service • Phases control arrivals • Each arrival is a new TCP socket (just like in the real world), picks a scenario, runs through it to completion or failure • Can reuse the same connection to run multiple scenarios with `loop`
You love Java Swing GUIs from the 90s • You need to benchmark a web server • You love Scala • Use Gatling (but make sure other devs, ops, and QA all know & love Scala too)
You love Java Swing GUIs from the 90s • You need to benchmark a web server • You love Scala • s/Scala/Python/ ? • Use Locust (same: make sure everyone knows Python. Also it’s HTTP-only)
You love Java Swing GUIs from the 90s • You need to benchmark a web server • You love Scala • s/Scala/Python/ ? • Otherwise, ARTILLERY :D • Everybody know JS, and JSON/YAML
load testing – being able to model the load accurately • Meaning – your load test needs to accurately simulate realistic conditions • If it doesn’t, you’re wasting your time
requests, something like wrk or loadtest is good: • wrk –c20 –d30s http://127.0.0.1/index.html • loadtest –c 20 –n 500 http://127.0.0.1/index.html • Web application or API – need scenarios, e.g.: • GET /search with some parameters • Pick one of the results • GET /products/$productId • POST /cart {productId: $productId}
requests, something like wrk or loadtest is good: • wrk –c20 –d30s http://127.0.0.1/index.html • loadtest –c 20 –n 500 http://127.0.0.1/index.html • Web application or API – need scenarios, e.g.: • GET /search with some parameters • Pick one of the results • GET /products/$productId • POST /cart {productId: $productId} • For this you want JMeter or Gatling or Artillery
• Behind a load-balancer? • How many TCP connections to open • Open all at once vs create new ones throughout the test • Throughput / load-distribution • For HTTP – requests • For WebSocket – messages/sec, pubsub patterns, active vs idle connections
• Behind a load-balancer? • How many TCP connections to open • Open all at once vs create new ones throughout the test • Throughput / load-distribution • For HTTP – requests • For WebSocket – messages/sec, pubsub patterns, active vs idle connections • Shape of peaks and valleys
time of this endpoint under heavy load, where heavy load = 25 RPS • How many active users can my server sustain, where a user = a WebSocket connection with 1 published message per second which is forwarded onto 75 other users on average
time of this endpoint under heavy load, where heavy load = 25 RPS • How many active users can my server sustain, where a user = a WebSocket connection with 1 published message per second which is forwarded onto 75 other users on average • How many concurrent user sessions above our current production peak can the production system sustain?
performance of the Express handler? (to compare with Rails) • Without network latency between client and Node.js server • Without network latency between Node.js and Rails
performance of the Express handler? (to compare with Rails) • Without network latency between client and Node.js server • Without network latency between Node.js and Rails • Without the time Rails takes to return the JSON
Vary them (different combos of query params) for a realistic simulation • Have the Node.js app record how long the request to Rails took and expose that as X- Backend-Response-Time header
Vary them (different combos of query params) for a realistic simulation • Have the Node.js app record how long the request to Rails took and expose that as X- Backend-Response-Time header • Use responseTime() middleware to have X- Response-Time
Vary them (different combos of query params) for a realistic simulation • Have the Node.js app record how long the request to Rails took and expose that as X-Backend- Response-Time header • Use responseTime() middleware to have X-Response- Time • For every request, the delta between the two is the time we spent purely in Express code – most of which is rendering React!
Load-testing an ecommerce app in production • Key: communication between dev, ops, and biz. • Start slowly in off-peak hours • Ramp up synthetic load to peak level during off hours
Load-testing an ecommerce app in production • Key: communication between dev, ops, and biz. • Start slowly in off-peak hours • Ramp up synthetic load to peak level during off hours • Start adding load during busy hours
Load-testing an ecommerce app in production • Key: communication between dev, ops, and biz. • Start slowly in off-peak hours • Ramp up synthetic load to peak level during off hours • Start adding load during busy hours • Gradually increase to e.g. 2X production
Load-testing an ecommerce app in production • Key: communication between dev, ops, and biz. • Start slowly in off-peak hours • Ramp up synthetic load to peak level during off hours • Start adding load during busy hours • Gradually increase to e.g. 2X production • Separate synthetic traffic (header, cookie)
Load-testing an ecommerce app in production • Key: communication between dev, ops, and biz. • Start slowly in off-peak hours • Ramp up synthetic load to peak level during off hours • Start adding load during busy hours • Gradually increase to e.g. 2X production • Separate synthetic traffic (header, cookie) • No-op certain transactions if needed (e.g. checking out)
• https://www.npmjs.com/package/0x • Works on both Linux and OSX (wraps perf or dtrace) • Generates nice flamegraphs • http://davidmarkclements.github.io/0x-demo/
scenarios, analyze request.js overhead, debug a memory leak in request, debug a memory leak in handlebars • IME heapdump is something you’ll reach out for more often
test • When you load-test, do it right • Model what actually happens in the app • Be methodical – hypothesis / question-driven • Learn how to instrument your apps • Profit!