Upgrade to Pro — share decks privately, control downloads, hide ads and more …

HubSpot Tech Talk - DIY Synthetic

HubSpot Tech Talk - DIY Synthetic

A talk I gave at a HubSpot tech talk in Boston.

Jonathan Klein

December 11, 2013
Tweet

More Decks by Jonathan Klein

Other Decks in Technology

Transcript

  1. About Me • Performance Engineer At Etsy • I write

    the Etsy Site Performance Reports Wednesday, December 11, 13
  2. About Me • Performance Engineer At Etsy • I write

    the Etsy Site Performance Reports • Organize Boston Web Perf Meetup Group Wednesday, December 11, 13
  3. Etsy Stats • World’s largest handmade marketplace • 1.5 billion

    page views/month Wednesday, December 11, 13
  4. Etsy Stats • World’s largest handmade marketplace • 1.5 billion

    page views/month • Almost $1B in sales last year Wednesday, December 11, 13
  5. Synthetic Monitoring Real User Monitoring Browser Instrumentation Navigation Timing API

    Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic “Real User Monitoring” Wednesday, December 11, 13
  6. Synthetic Monitoring Real User Monitoring Browser Instrumentation Navigation Timing API

    Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic “Real User Monitoring” Wednesday, December 11, 13
  7. Synthetic Monitoring Real User Monitoring Browser Instrumentation Navigation Timing API

    Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic “Real User Monitoring” Wednesday, December 11, 13
  8. Synthetic Monitoring Real User Monitoring Browser Instrumentation Navigation Timing API

    Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic “Real User Monitoring” Wednesday, December 11, 13
  9. Synthetic Monitoring Real User Monitoring Browser Instrumentation Navigation Timing API

    Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic “Real User Monitoring” Wednesday, December 11, 13
  10. Synthetic Monitoring Real User Monitoring Browser Instrumentation Navigation Timing API

    Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic “Real User Monitoring” Wednesday, December 11, 13
  11. Why We Use It • Wanted long term synthetic trending

    • Automated tests Wednesday, December 11, 13
  12. Why We Use It • Wanted long term synthetic trending

    • Automated tests • More control Wednesday, December 11, 13
  13. Why We Use It • Wanted long term synthetic trending

    • Automated tests • More control • Low cost Wednesday, December 11, 13
  14. { "server": "http://www.yourwptinstall.com", "pending_dir": "private", "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"], "urls": {

    "label": "http://www.example.com", "other_label": "http://www.anotherexample.com" }, "graphite": "graphite.example.com", "logging_ns": "webpagetest.private", "splunkLog": "/var/log/webpagetest/results.log" } Wednesday, December 11, 13
  15. { "server": "http://www.yourwptinstall.com", "pending_dir": "private", "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"], "urls": {

    "label": "http://www.example.com", "other_label": "http://www.anotherexample.com" }, "graphite": "graphite.example.com", "logging_ns": "webpagetest.private", "splunkLog": "/var/log/webpagetest/results.log" } Wednesday, December 11, 13
  16. { "server": "http://www.yourwptinstall.com", "pending_dir": "private", "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"], "urls": {

    "label": "http://www.example.com", "other_label": "http://www.anotherexample.com" }, "graphite": "graphite.example.com", "logging_ns": "webpagetest.private", "splunkLog": "/var/log/webpagetest/results.log" } Wednesday, December 11, 13
  17. { "server": "http://www.yourwptinstall.com", "pending_dir": "private", "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"], "urls": {

    "label": "http://www.example.com", "other_label": "http://www.anotherexample.com" }, "graphite": "graphite.example.com", "logging_ns": "webpagetest.private", "splunkLog": "/var/log/webpagetest/results.log" } Wednesday, December 11, 13
  18. { "server": "http://www.yourwptinstall.com", "pending_dir": "private", "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"], "urls": {

    "label": "http://www.example.com", "other_label": "http://www.anotherexample.com" }, "graphite": "graphite.example.com", "logging_ns": "webpagetest.private", "splunkLog": "/var/log/webpagetest/results.log" } Wednesday, December 11, 13
  19. # Run Test */25 * * * * php run.php

    Wednesday, December 11, 13
  20. # Run Test */25 * * * * php run.php

    -c foo.conf Wednesday, December 11, 13
  21. # Run Test */25 * * * * php run.php

    -c foo.conf # Get Results * * * * * php get_results.php Wednesday, December 11, 13
  22. { "server": "http://example.com", "pending_dir": "private", "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"], "urls": {

    "label": "http://www.example.com", "other_label": "http://www.webpagetest.org" }, "graphite": "graphite.example.com", "logging_ns": "webpagetest.private", "splunkLog": "/var/log/webpagetest/results.log", "prepend": "Login", "username": "someuser", "password": "somepassword", "run_options": { "video": 0 } } Wednesday, December 11, 13
  23. { "server": "http://example.com", "pending_dir": "private", "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"], "urls": {

    "label": "http://www.example.com", "other_label": "http://www.webpagetest.org" }, "graphite": "graphite.example.com", "logging_ns": "webpagetest.private", "splunkLog": "/var/log/webpagetest/results.log", "prepend": ["BlockThirdParty", "bypass_cdn.txt"], "run_options": { "video": 0 } } Wednesday, December 11, 13
  24. foreach ($prepends as $prepend) { if (file_exists($scripts_dir . $prepend)) {

    $script .= file_get_contents($scripts_dir . $prepend); } elseif (method_exists($this, $prepend)) { $script .= $this->$prepend(); } } Wednesday, December 11, 13
  25. setDns site.etsystatic.com 38.106.64.123 setDns img0.etsystatic.com 38.106.64.125 setDns img1.etsystatic.com 38.106.64.125 setDns

    ny-image0.etsy.com 38.106.64.125 setDns ny-image1.etsy.com 38.106.64.125 bypass_cdn.txt Wednesday, December 11, 13
  26. <?php // Splunk $splunkLogger = new SplunkLogger($config['splunkLog'], $logging_ns); foreach ($results

    as $result) { $splunkLogger->log($result); } // Graphite $grapher = new Grapher($config['graphite'], $logging_ns); $grapher->graphResults($results); Wednesday, December 11, 13
  27. Disadvantages vs. Paid Tools • Harder to set up/maintain •

    Functionality Wednesday, December 11, 13
  28. Disadvantages vs. Paid Tools • Harder to set up/maintain •

    Functionality • Regions Wednesday, December 11, 13
  29. Disadvantages vs. Paid Tools • Harder to set up/maintain •

    Functionality • Regions • Full page oriented Wednesday, December 11, 13
  30. Disadvantages vs. Paid Tools • Harder to set up/maintain •

    Functionality • Regions • Full page oriented • Less consistent Wednesday, December 11, 13
  31. Advantages vs. Paid Tools • Cost • Flexibility • People

    know the tool Wednesday, December 11, 13
  32. Advantages vs. Paid Tools • Cost • Flexibility • People

    know the tool • Multi-page flows Wednesday, December 11, 13
  33. Advantages vs. Paid Tools • Cost • Flexibility • People

    know the tool • Multi-page flows • Scripting Wednesday, December 11, 13
  34. WebPagetest Catchpoint • Multi-page flows • Scripted tests • High

    volume testing • Ad-hoc tests Wednesday, December 11, 13
  35. WebPagetest Catchpoint • Multi-page flows • Scripted tests • High

    volume testing • Ad-hoc tests • DNS Wednesday, December 11, 13
  36. WebPagetest Catchpoint • Multi-page flows • Scripted tests • High

    volume testing • Ad-hoc tests • DNS • API Wednesday, December 11, 13
  37. WebPagetest Catchpoint • Multi-page flows • Scripted tests • High

    volume testing • Ad-hoc tests • DNS • API • Single Object Tests Wednesday, December 11, 13
  38. WebPagetest Catchpoint • Multi-page flows • Scripted tests • High

    volume testing • Ad-hoc tests • DNS • API • Single Object Tests • Multiple locations Wednesday, December 11, 13
  39. WebPagetest Catchpoint • Multi-page flows • Scripted tests • High

    volume testing • Ad-hoc tests • DNS • API • Single Object Tests • Multiple locations • Comparisons Wednesday, December 11, 13
  40. Improving the Process • Manage disk space • Multiple regions

    • More canned scripts Wednesday, December 11, 13
  41. Improving the Process • Manage disk space • Multiple regions

    • More canned scripts • Automate upgrades Wednesday, December 11, 13
  42. Speed Index • Synthetic only • Used heavily inside Google

    • The best metric for perceived load time Wednesday, December 11, 13
  43. Speed Index • Synthetic only • Used heavily inside Google

    • The best metric for perceived load time • (this is what we care about) Wednesday, December 11, 13
  44. WebPagetest + Vagrant • Install VirtualBox • Install Vagrant •

    https://gist.github.com/jklein/6992296 Wednesday, December 11, 13
  45. WebPagetest + Vagrant • Install VirtualBox • Install Vagrant •

    https://gist.github.com/jklein/6992296 •vagrant up Wednesday, December 11, 13
  46. WebPagetest + Vagrant • Install VirtualBox • Install Vagrant •

    https://gist.github.com/jklein/6992296 •vagrant up • http://localhost:8080 Wednesday, December 11, 13
  47. WebPagetest Relay -- locations.ini [WPT_Dulles_IE8] browser=IE 8 label="WebPagetest.org Dulles, VA

    - IE8" relayServer="http://www.webpagetest.org/" relayKey=<your API key> relayLocation=Dulles_IE8 Wednesday, December 11, 13