Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Unpacking the Black Box: Benchmarking JS Parsin...

Daniel Espeset
September 17, 2014

Unpacking the Black Box: Benchmarking JS Parsing and Execution on Mobile Devices

Optimizing the experience on mobile devices is a major priority for our industry, but many client performance characteristics are not well understood. In pursuit of the 1000ms time-to-glass holy grail, we wrote a tool to benchmark the initial parse and execute times of our JavaScript files. While high end devices are expectedly fast, midrange and lower end ones are surprisingly slow. We’ll share our full suite of results, the methodology used and some optimization techniques – plus we released the tool used to do these tests: github.com/etsy/DeviceTiming

Daniel Espeset

September 17, 2014
Tweet

More Decks by Daniel Espeset

Other Decks in Programming

Transcript

  1. Daniel Espeset @danielespeset Velocity NYC – September 17, 2014 Benchmarking

    JS Parsing and Execution on Mobile Devices Unpacking the Black Box
  2. Daniel Espeset @danielespeset How long does our JavaScript payload take

    to parse and execute, once it hits the browser? 18
  3. Adding naive timers to our JS bundle var start =

    new Date().getTime(); // ... bulk of your code var duration = new Date().getTime() - start; 1 - 600 21
  4. var start = new Date().getTime(); // ... bulk of your

    code var duration = new Date().getTime() - start; 1 - 600 24 A bunch of work happens before this runs. Adding naive timers to our JS bundle
  5. Daniel Espeset @danielespeset Next questions 26 1. How long does

    it take to parse? 2. Can we measure this on every device in our lab?
  6. We could wrap timers around a <script> tag <script>var beforeJquery

    = new Date().getTime()</script> <script src=”jquery.js”></script> <script>var took = new Date().getTime() - beforeJquery<script> 20 21 22 32
  7. Same idea, but in a loop var runs = 1000;

    results = []; while (runs--) { d.write(“<script>var start = Date.now();</script>”); d.write(“<script src=‘jQuery.js’></script>”); d.write(“<script>”); d.write(“results.push(Date.now() - start);”); d.write(“var e = d.getElementById(‘test’);”); d.write(“e.parentNode.removeChild(e);”); d.write(“</script>”); } 22 23 24 25 26 27 28 29 30 31 32 34
  8. Daniel Espeset @danielespeset Carlos Bueno did this in 2010 “Measuring

    Javascript Parse and Load” bit.ly/js-parse-and-load 35
  9. Step 1. “window scoper” transforms implicit globals to explicit ones.

    var foo = {}; (function() { // ... }); 1 2 - 99 38 window.foo = {}; (function() { // ... }); 1 2 - 99
  10. Step 2. wrap the contents in a named function 39

    window.foo = {}; (function() { // ... }); 1 2 - 99 function sourceCode() { window.foo = {}; (function() { // ... }); } 1 2 3 - 100 101
  11. Step 3. turn the function into a string, eval and

    execute it var start = new Date().getTime(); eval(“function sourceCode() { window.foo = {}; ... ”); var parse = new Date().getTime(); var parseTook = start - parse; sourceCode(); var execTook = new Date().getTime() - parse; 1 2 3 4 5 6 40
  12. Step 3. turn the function into a string, eval and

    execute it var start = new Date().getTime(); eval(“function sourceCode() { window.foo = {}; ... ”); var parse = new Date().getTime(); var parseTook = start - parse; sourceCode(); var execTook = new Date().getTime() - parse; 1 2 3 4 5 6 41
  13. Step 3. turn the function into a string, eval and

    execute it var start = new Date().getTime(); eval(“function sourceCode() { window.foo = {}; ... ”); var parse = new Date().getTime(); var parseTook = start - parse; sourceCode(); var execTook = new Date().getTime() - parse; 1 2 3 4 5 6 42
  14. Daniel Espeset @danielespeset 44 results for jquery 1.8.2 + almond

    Apple (Macbook Pro) Chrome 31 RAM: 16GB CPU: 2.4GHz GPU: 1058MHz 36ms total
  15. Daniel Espeset @danielespeset 45 results for jquery 1.8.2 + almond

    319ms total Samsung Galaxy S3 Mobile Chrome 31 RAM: 1GB CPU: 1.4GHz GPU: 400MHz
  16. Daniel Espeset @danielespeset 46 results for jquery 1.8.2 + almond

    1389ms total LG Optimus V (VM670) Android Browser 4.0 RAM: 512MB CPU: 600MHz GPU: 128MHz
  17. Daniel Espeset @danielespeset 47 1. How long to eval a

    big function with our code 2. How long to execute it What does this measure?
  18. Daniel Espeset @danielespeset 48 1. Eval is a reasonable proxy

    for <script> 2. This effectively separates parse and exec Assumptions
  19. Add timers to the top and bottom of the big

    wrapper function var start = new Date().getTime(), innerStart, innerEnd; eval(“function sourceCode() { innerStart = new Date().getTime(); window.foo = {}; ... }”); var execStart = new Date().getTime(); sourceCode(); var execEnd = new Date().getTime() - parse; 1 2 3 4 5 6 7 8 49
  20. Add a timer to the top of the big wrapper

    function 50 var start = new Date().getTime(), innerStart, innerEnd; eval(“function sourceCode() { innerStart = new Date().getTime(); window.foo = {}; ... }”); var execStart = new Date().getTime(); sourceCode(); var execEnd = new Date().getTime() - parse; 1 2 3 4 5 6 7 8
  21. (what this looks like at runtime) 51 var execStart =

    new Date().getTime(); (function sourceCode() { innerStart = new Date().getTime(); window.foo = {}; // ... })(); var execEnd = new Date().getTime() - parse; 1 2 3 4 5 6 7 8 The time between these should be ~0ms
  22. (what this looks like at runtime) 52 var execStart =

    new Date().getTime(); (function sourceCode() { innerStart = new Date().getTime(); window.foo = {}; // ... })(); var execEnd = new Date().getTime() - parse; 1 2 3 4 5 6 7 8 Chrome on Samsung Galaxy S3 50–100ms
  23. Step 3 REDUX: turn the function into a string, eval

    it var start = new Date().getTime(); eval(“var parse = new Date().getTime(); !function(){ ... }()”); var execTook = new Date().getTime() - parse; var parseTook = start - parse; 1 2 3 4 55
  24. Parse: the time between start, and the first line of

    the eval running. var start = new Date().getTime(); eval(“var parse = new Date().getTime(); !function(){ ... }()”); var execTook = new Date().getTime() - parse; var parseTook = start - parse; 1 2 3 4 56
  25. Exec: the time between the first line of the eval,

    and the line after var start = new Date().getTime(); eval(“var parse = new Date().getTime(); !function(){ ... }()”); var execTook = new Date().getTime() - parse; var parseTook = start - parse; 1 2 3 4 57