Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DIY Synthetic: Private WebPagetest Magic

DIY Synthetic: Private WebPagetest Magic

I gave this talk at Velocity NY 2013, and it covers the installation and use of a private WebPagetest instance, as well as the open source tool wpt-script which Etsy built.

Jonathan Klein

October 16, 2013
Tweet

More Decks by Jonathan Klein

Other Decks in Technology

Transcript

  1. About Me • Performance Engineer At Etsy • I write

    the Etsy Site Performance Reports Wednesday, October 16, 13
  2. About Me • Performance Engineer At Etsy • I write

    the Etsy Site Performance Reports • Organize Boston Web Perf Meetup Group Wednesday, October 16, 13
  3. Etsy Stats • World’s largest handmade marketplace • 1.5 billion

    page views/month • Almost $1B in sales last year Wednesday, October 16, 13
  4. Synthetic Monitoring Real User Monitoring Browser Instrumentation Navigation Timing API

    Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic “Real User Monitoring” Wednesday, October 16, 13
  5. Synthetic Monitoring Real User Monitoring Browser Instrumentation Navigation Timing API

    Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic “Real User Monitoring” Wednesday, October 16, 13
  6. Synthetic Monitoring Real User Monitoring Browser Instrumentation Navigation Timing API

    Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic “Real User Monitoring” Wednesday, October 16, 13
  7. Synthetic Monitoring Real User Monitoring Browser Instrumentation Navigation Timing API

    Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic “Real User Monitoring” Wednesday, October 16, 13
  8. Synthetic Monitoring Real User Monitoring Browser Instrumentation Navigation Timing API

    Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic “Real User Monitoring” Wednesday, October 16, 13
  9. Synthetic Monitoring Real User Monitoring Browser Instrumentation Navigation Timing API

    Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic “Real User Monitoring” Wednesday, October 16, 13
  10. Why We Use It • Wanted long term synthetic trending

    • Automated tests Wednesday, October 16, 13
  11. Why We Use It • Wanted long term synthetic trending

    • Automated tests • More control Wednesday, October 16, 13
  12. Why We Use It • Wanted long term synthetic trending

    • Automated tests • More control • Low cost Wednesday, October 16, 13
  13. { "server": "http://www.yourwptinstall.com", "pending_dir": "private", "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"], "urls": {

    "label": "http://www.example.com", "other_label": "http://www.anotherexample.com" }, "graphite": "graphite.example.com", "logging_ns": "webpagetest.private", "splunkLog": "/var/log/webpagetest/results.log" } Wednesday, October 16, 13
  14. { "server": "http://www.yourwptinstall.com", "pending_dir": "private", "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"], "urls": {

    "label": "http://www.example.com", "other_label": "http://www.anotherexample.com" }, "graphite": "graphite.example.com", "logging_ns": "webpagetest.private", "splunkLog": "/var/log/webpagetest/results.log" } Wednesday, October 16, 13
  15. { "server": "http://www.yourwptinstall.com", "pending_dir": "private", "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"], "urls": {

    "label": "http://www.example.com", "other_label": "http://www.anotherexample.com" }, "graphite": "graphite.example.com", "logging_ns": "webpagetest.private", "splunkLog": "/var/log/webpagetest/results.log" } Wednesday, October 16, 13
  16. { "server": "http://www.yourwptinstall.com", "pending_dir": "private", "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"], "urls": {

    "label": "http://www.example.com", "other_label": "http://www.anotherexample.com" }, "graphite": "graphite.example.com", "logging_ns": "webpagetest.private", "splunkLog": "/var/log/webpagetest/results.log" } Wednesday, October 16, 13
  17. { "server": "http://www.yourwptinstall.com", "pending_dir": "private", "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"], "urls": {

    "label": "http://www.example.com", "other_label": "http://www.anotherexample.com" }, "graphite": "graphite.example.com", "logging_ns": "webpagetest.private", "splunkLog": "/var/log/webpagetest/results.log" } Wednesday, October 16, 13
  18. # Run Test */25 * * * * php run.php

    Wednesday, October 16, 13
  19. # Run Test */25 * * * * php run.php

    -c foo.conf Wednesday, October 16, 13
  20. # Run Test */25 * * * * php run.php

    -c foo.conf # Get Results * * * * * php get_results.php Wednesday, October 16, 13
  21. { "server": "http://example.com", "pending_dir": "private", "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"], "urls": {

    "label": "http://www.example.com", "other_label": "http://www.webpagetest.org" }, "graphite": "graphite.example.com", "logging_ns": "webpagetest.private", "splunkLog": "/var/log/webpagetest/results.log", "prepend": "Login", "username": "someuser", "password": "somepassword", "run_options": { "video": 0 } } Wednesday, October 16, 13
  22. { "server": "http://example.com", "pending_dir": "private", "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"], "urls": {

    "label": "http://www.example.com", "other_label": "http://www.webpagetest.org" }, "graphite": "graphite.example.com", "logging_ns": "webpagetest.private", "splunkLog": "/var/log/webpagetest/results.log", "prepend": ["BlockThirdParty", "bypass_cdn.txt"], "run_options": { "video": 0 } } Wednesday, October 16, 13
  23. foreach ($prepends as $prepend) { if (file_exists($scripts_dir . $prepend)) {

    $script .= file_get_contents($scripts_dir . $prepend); } elseif (method_exists($this, $prepend)) { $script .= $this->$prepend(); } } Wednesday, October 16, 13
  24. setDns site.etsystatic.com 38.106.64.123 setDns img0.etsystatic.com 38.106.64.125 setDns img1.etsystatic.com 38.106.64.125 setDns

    img2.etsystatic.com 38.106.64.125 setDns img3.etsystatic.com 38.106.64.125 setDns ny-image0.etsy.com 38.106.64.125 setDns ny-image1.etsy.com 38.106.64.125 setDns ny-image2.etsy.com 38.106.64.125 setDns ny-image3.etsy.com 38.106.64.125 bypass_cdn.txt Wednesday, October 16, 13
  25. <?php // Splunk $splunkLogger = new SplunkLogger($config['splunkLog'], $logging_ns); foreach ($results

    as $result) { $splunkLogger->log($result); } // Graphite $grapher = new Grapher($config['graphite'], $logging_ns); $grapher->graphResults($results); Wednesday, October 16, 13
  26. Disadvantages vs. Paid Tools • Harder to set up/maintain •

    Functionality Wednesday, October 16, 13
  27. Disadvantages vs. Paid Tools • Harder to set up/maintain •

    Functionality • Regions Wednesday, October 16, 13
  28. Disadvantages vs. Paid Tools • Harder to set up/maintain •

    Functionality • Regions • Full page oriented Wednesday, October 16, 13
  29. Disadvantages vs. Paid Tools • Harder to set up/maintain •

    Functionality • Regions • Full page oriented • Less consistent Wednesday, October 16, 13
  30. Advantages vs. Paid Tools • Cost • Flexibility • People

    know the tool Wednesday, October 16, 13
  31. Advantages vs. Paid Tools • Cost • Flexibility • People

    know the tool • Multi-page flows Wednesday, October 16, 13
  32. Advantages vs. Paid Tools • Cost • Flexibility • People

    know the tool • Multi-page flows • Scripting Wednesday, October 16, 13
  33. WebPagetest Catchpoint • Multi-page flows • Scripted tests • High

    volume testing • Ad-hoc tests Wednesday, October 16, 13
  34. WebPagetest Catchpoint • Multi-page flows • Scripted tests • High

    volume testing • Ad-hoc tests • DNS Wednesday, October 16, 13
  35. WebPagetest Catchpoint • Multi-page flows • Scripted tests • High

    volume testing • Ad-hoc tests • DNS • API Wednesday, October 16, 13
  36. WebPagetest Catchpoint • Multi-page flows • Scripted tests • High

    volume testing • Ad-hoc tests • DNS • API • Single Object Tests Wednesday, October 16, 13
  37. WebPagetest Catchpoint • Multi-page flows • Scripted tests • High

    volume testing • Ad-hoc tests • DNS • API • Single Object Tests • Multiple locations Wednesday, October 16, 13
  38. WebPagetest Catchpoint • Multi-page flows • Scripted tests • High

    volume testing • Ad-hoc tests • DNS • API • Single Object Tests • Multiple locations • Comparisons Wednesday, October 16, 13
  39. Improving the Process • Manage disk space • Multiple regions

    • More canned scripts Wednesday, October 16, 13
  40. Improving the Process • Manage disk space • Multiple regions

    • More canned scripts • Automate upgrades Wednesday, October 16, 13
  41. WebPagetest + Vagrant • Install VirtualBox • Install Vagrant •

    https://gist.github.com/jklein/6992296 Wednesday, October 16, 13
  42. WebPagetest + Vagrant • Install VirtualBox • Install Vagrant •

    https://gist.github.com/jklein/6992296 •vagrant up Wednesday, October 16, 13
  43. WebPagetest + Vagrant • Install VirtualBox • Install Vagrant •

    https://gist.github.com/jklein/6992296 •vagrant up • http://localhost:8080 Wednesday, October 16, 13