Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Performance Testing & Profiling: A Virtuous Cy...

Performance Testing & Profiling: A Virtuous Cycle by Dan Crosta

Donald Knuth famously said that we should avoid optimization 97% of the time and focus on the "critical 3%". How can we identify that 3%? How can we best focus our optimization efforts, and avoid the "root of all evil" that is premature optimization? This talk introduces key types of performance testing, and demonstrates how they can be paired with profiling techniques in a cycle of improvement.

PyCon 2014

April 12, 2014
Tweet

More Decks by PyCon 2014

Other Decks in Programming

Transcript

  1. Dan Crosta | @lazlofruvous Performance Testing & Profiling: A Virtuous

    Cycle Dan Crosta | Magnetic http://late.am/ | @lazlofruvous
  2. Dan Crosta | @lazlofruvous Overview • Performance testing web apps

    • Profiling with the standard library • Instrumentation • The Virtuous Cycle
  3. Dan Crosta | @lazlofruvous Performance Testing Basics • Generate requests

    against your app • Record and replay production • Measure response time & error rate
  4. Dan Crosta | @lazlofruvous Types of Performance Testing • Stress

    Test • Load Test ! (I won’t talk about) • Spike Test • Soak Test
  5. Dan Crosta | @lazlofruvous Stress Test • Generate excessive load

    • Lots of requests • Slow/difficult requests • Adversarial testing • “How much?” • Identify breaking point • Especially if you control synthetic load
  6. Dan Crosta | @lazlofruvous Load Test • Generate specific, constant

    load • Expected conditions • Exaggerated conditions • “What If?” • Capacity planning
  7. Dan Crosta | @lazlofruvous Best Practices • Isolate testing from

    external influences • Use dedicated load testing environment • “Scaled down” copies of all components
  8. Dan Crosta | @lazlofruvous Best Practices • Generate load consistently

    • Random considered harmful • Automate, automate, automate
  9. Dan Crosta | @lazlofruvous Profiling • Batteries included • cProfile,

    pstats • Documentation not really included • Goofy, horrible API • Avoid run(), runctx()
  10. Dan Crosta | @lazlofruvous Profiling import cProfile ! profiler =

    cProfile.Profile()
 profiler.enable() ! # do stuff here ... ! profiler.disable() profiler.dump_stats("myprogram.prof")
  11. Dan Crosta | @lazlofruvous Profiling import pstats ! stats =

    pstats.Stats("myprogram.prof") stats.sort_stats("calls") stats.print_stats() ! # prints a lot of stuff ...
  12. Dan Crosta | @lazlofruvous Profiling in Practice • “Why is

    it slow?” • Good for identifying un-optimized code • Tight loops, recursion, lots of function calls • These are candidates for optimization • Good for identifying bottlenecks • Distinguish between slow external resource and slow application code
  13. Dan Crosta | @lazlofruvous Instrumentation • Use statsd to collect

    time-series metrics • Lightweight, low-overhead, always-on profiling • Two key instruments: • Counters let you know how many things happened • Timers let you know how long they each took
  14. Dan Crosta | @lazlofruvous Instrumentation from statsd import statsd !

    @statsd.timed("login.response_time") def handle_login(username, password): if authenticated(username, password): statsd.increment("login.valid", 1) return redirect("/homepage") else: saatsd.increment("login.invalid", 1) return redirect("/login")
  15. Dan Crosta | @lazlofruvous Instrumentation • Learn what’s normal for

    your app • Bonus: alert when things are not normal • “Does the real world match expectations?”
  16. Dan Crosta | @lazlofruvous The Virtuous Cycle Instrumentation
 & Alerting

    Performance Testing Profiling Performance Optimization