Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Performance Testing Modern Apps

Dustin Whittle
September 09, 2015

Performance Testing Modern Apps

Performance Testing Modern Apps presented at DjangoCon 2015

Dustin Whittle

September 09, 2015
Tweet

More Decks by Dustin Whittle

Other Decks in Technology

Transcript

  1. Dus-n Whi8le • dus-nwhi8le.com • @dus-nwhi8le • San Francisco, California,

    USA • Technologist, Traveler, Pilot, Skier, Diver, Sailor, Golfer
  2. MicrosoI found that Bing searches that were 2 seconds slower

    resulted in a 4.3% drop in revenue per user
  3. How fast is fast enough? § Performance is key to

    a great user experience - Under 100ms is perceived as reac$ng instantaneously - A 100ms to 300ms delay is percep$ble - 1 second is about the limit for the user's flow of thought to stay uninterrupted - Users expect a site to load in 2 seconds - AIer 3 seconds, 40% will abandon your site. - 10 seconds is about the limit for keeping the user's a*en$on § Modern applica-ons spend more -me in the browser than on the server-side
  4. Login Flight Status Search Flight Purchase Mobile Big data SOA

    NOSQL Cloud Agile Web Application complexity is exploding
  5. Benchmarking acmedemoapp.com (be patient) Finished 187 requests Server Software: Apache-Coyote/1.1

    Server Hostname: acmedemoapp.com Server Port: 80 Document Path: / Document Length: 5217 bytes Concurrency Level: 1 Time taken for tests: 10.039 seconds Complete requests: 187 Failed requests: 0 Keep-Alive requests: 187 Total transferred: 1021768 bytes HTML transferred: 975579 bytes Requests per second: 18.63 [#/sec] (mean) Time per request: 53.687 [ms] (mean)
  6. Benchmarking acmedemoapp.com (be patient) Finished 659 requests Server Software: Apache-Coyote/1.1

    Server Hostname: acmedemoapp.com Server Port: 80 Document Path: / Document Length: 5217 bytes Concurrency Level: 10 Time taken for tests: 10.015 seconds Complete requests: 659 Failed requests: 0 Keep-Alive requests: 659 Total transferred: 3600776 bytes HTML transferred: 3438003 bytes Requests per second: 65.80 [#/sec] (mean) Time per request: 151.970 [ms] (mean) Time per request: 15.197 [ms] (mean, across all
  7. ** SIEGE 3.0.6 ** Preparing 10 concurrent users for battle.

    The server is now under siege... Lifting the server siege... done. Transactions: 623 hits Availability: 100.00 % Elapsed time: 9.57 secs Data transferred: 3.10 MB Response time: 0.15 secs Transaction rate: 65.10 trans/sec Throughput: 0.32 MB/sec Concurrency: 9.91 Successful transactions: 623 Failed transactions: 0 Longest transaction: 0.30 Shortest transaction: 0.10
  8. SPROXY v1.02 listening on port 9001 ...appending HTTP requests to:

    ./urls.txt ...default connection timeout: 120 seconds
  9. wget -r -l 0 -t 1 --spider -w 1 -e

    "http_proxy = http://127.0.0.1:9001" "http://acmedemoapp.com/" sort -u -o urls.txt urls.txt
  10. ** SIEGE 3.0.6 ** Preparing 10 concurrent users for battle.

    The server is now under siege... Lifting the server siege... done. Transactions: 623 hits Availability: 100.00 % Elapsed time: 9.57 secs Data transferred: 3.10 MB Response time: 0.15 secs Transaction rate: 65.10 trans/sec Throughput: 0.32 MB/sec Concurrency: 9.91 Successful transactions: 623 Failed transactions: 0 Longest transaction: 0.30 Shortest transaction: 0.10
  11. import mechanize import time class Transaction(object): def run(self): br =

    mechanize.Browser() br.set_handle_robots(False) start_timer = time.time() resp = br.open(‘http://acmedemoapp.com/) resp.read() latency = time.time() - start_timer self.custom_timers['homepage'] = latency start_timer = time.time() resp = br.open(‘http://acmedemoapp.com/cart') resp.read() latency = time.time() - start_timer self.custom_timers['cart'] = latency assert (resp.code == 200)
  12. A utility for arming (creating)  many bees (micro EC2

    instances) to attack (load test) targets (web applications)
  13. bees up -s 2 -g default -z us-west-2b -i ami-bc05898c

    -k appdynamics- dustinwhittle-aws-us-west-2 -l ec2-user
  14. Connecting to the hive. Attempting to call up 2 bees.

    Waiting for bees to load their machine guns... . . . . Bee i-3828400c is ready for the attack. Bee i-3928400d is ready for the attack. The swarm has assembled 2 bees.
  15. Read 2 bees from the roster. Bee i-3828400c: running @

    54.212.22.176 Bee i-3928400d: running @ 50.112.6.191
  16. Read 2 bees from the roster. Connecting to the hive.

    Assembling bees. Each of 2 bees will fire 500 rounds, 25 at a time. Stinging URL so it will be cached for the attack. Organizing the swarm. … Offensive complete. Complete requests: 1000 Requests per second: 306.540000 [#/sec] (mean) Time per request: 163.112000 [ms] (mean) 50% response time: 151.000000 [ms] (mean) 90% response time: 192.000000 [ms] (mean) Mission Assessment: Target crushed bee offensive. The swarm is awaiting new orders.
  17. Read 2 bees from the roster. Connecting to the hive.

    Assembling bees. Each of 2 bees will fire 50000 rounds, 500 at a time. Stinging URL so it will be cached for the attack. Organizing the swarm. … Offensive complete. Complete requests: 100000 Requests per second: 502.420000 [#/sec] (mean) Time per request: 360.114000 [ms] (mean) 50% response time: 451.000000 [ms] (mean) 90% response time: 402.000000 [ms] (mean) Mission Assessment: Target crushed bee offensive. The swarm is awaiting new orders.
  18. Read 2 bees from the roster. Connecting to the hive.

    Assembling bees. Each of 2 bees will fire 50000 rounds, 500 at a time. Stinging URL so it will be cached for the attack. Organizing the swarm. Bee 0 is joining the swarm. Bee 1 is joining the swarm. Bee 0 is firing his machine gun. Bang bang! Bee 0 lost sight of the target (connection timed out). Bee 1 lost sight of the target (connection timed out). Offensive complete. Target timed out without fully responding to 2 bees. No bees completed the mission. Apparently your bees are peace-loving hippies. The swarm is awaiting new orders.
  19. Read 2 bees from the roster. Connecting to the hive.

    Calling off the swarm. Stood down 2 bees.
  20. Test for failures • NetFlix Simian Army + Chaos Monkey


    • What happens if you lose a caching layer? • What happens if dependencies slow down?
  21. Performance Ma*ers • Treat performance as a feature • Using

    the 14kb Rule for instant loading • Markup management • Elimina$ng excess AJAX calls • Working with and around applica$on cache • Developing a responsive design + image strategy • Implemen$ng a good touch-first strategy • Code management for good produc$on and development experiences • Using task runners to build and deploy produc$on code
  22. Best Prac$ces • Treat performance as a feature • Capacity

    plan and load test the server-side • Op-mize and performance test the client-side • Understand your star-ng point • Instrument everything • Monitor performance in development and produc-on • Measure the difference of every change • Automate performance tes-ng in your build and deployment process • Understand how failures impact performance
  23. The protocols are evolving • The limita$ons of HTTP/1.X forced

    us to develop various applica$on workarounds (sharding, concatena$on, spri$ng, inlining, etc.) to op$mize performance. However, in the process we’ve also introduced numerous regressions: poor caching, unnecessary downloads, delayed execu$on, and more. • HTTP/2 eliminates the need for these hacks and allows us to both simplify our applica$ons and deliver improved performance. • You should unshard, unconcat, and unsprite your assets • You should switch from inlining to server push • Read Ilya Grigorik awesome book on browser performance - h*p://hpbn.co/ h*p2