Capturing Speed of User Experience using UserTiming API

Sergey Chernyshev talks about the challenges of capturing accurate metrics around speed of user experience and how instrumenting your application using UserTiming API can help.

He gives a brief overview of automated metrics that are captured by the browser and shows how custom metrics, specific to features of your user interface can help your teams better relate to these numbers and help more directly align them with user interface optimizations.

Sergey shows how to use native UserTiming API to mark specific events and measure differences between these marks on Performance Timeline.

He also introduces UX-Capture JavaScript library to help simplify and unify this instrumentation across your site. We also look at debugging process using Chrome Developer Tools and how metrics captured with UserTiming API can be automatically picked up by open WebPageTest tool and other synthetic and RUM monitoring tools.

Transcript

1. Capturing Speed of User Experience With UserTiming API November 27, 2018 @ NY Web Performance Meetup Sergey Chernyshev
2. Sergey Chernyshev Principal Engineer @ Meetup November 27, 2018 @ NY Web Performance Meetup • Care about
 UX speed • Organize
 NY Web Performance Meetup Group • Work on
 Web Platform team @ Meetup
3. • Monitor for degradation • Analyze code for speed issues • Verify experience improvements • Prioritize improvements • Budget for speed initiatives Why do we measure speed? Operations Developers Developers
 Business
 Business
4. • Pinterest: -40% wait time => +15% SEO, +15% conversion (2017) • DoubleClick: 5s vs 19s on mobile => 2x ad revenue (2016) • Trainline: -300ms latency => +$11.5M / year revenue (2016) • Etsy: +160Kb => +12% bounce rate (2014) • Edmunds: -77% load time => +20% page views (2011) • Mozilla: -2.2s => +15.4% Downloads (2010) • Google: +400ms => -0.21% searches after experiment! (2009) • Netflix: +GZip => -43% Traffic cost (2008) • Amazon: +100ms => -1% revenue (2008) • Google: +500ms => -25% searches (2006) Stats @ WPOstats.com
5. Synthetic • Tests from
 particular location • Tester controls instrumentation • One metric value • Data can have lots of details for analysis
6. Real User Measurement • Real users (a lot of them) • A lot of data (need to store it) • All noise you can get, requires filtering • Metrics are distributions • Can correlate to business KPIs
7. Measure User Experience Not Technology
8. Measure
 User Experience • Great experience for users • "Fast" is just a component • Correlate what you measure to business KPIs • Do not measure what's easy, measure what matters
9. • DNS, SSL/TLS, Time To First Byte (TTFB) • Page Load, Document Complete, Fully Loaded • First Paint & First Contentful Paint (FCP) • Above the Fold Time (AFT) • Speed Index Today’s Metrics
10. • Shows when completely useless part is over Time To First Paint TTFP: 3.5s
11. • When everything is finally visible Above the Fold Time AFT: 15.3s
12. Speed Index • When everything is finally visible Speed Index: 8618
13. • “Time To First Tweet” UX Speed • “Pinner Wait Time” Improving performance on twitter.com https://blog.twitter.com/engineering/en_us/a/2012/ improving-performance-on-twittercom.html Four lessons in making Pinterest faster on Android https://medium.com/@Pinterest_Engineering/four- lessons-in-making-pinterest-faster-on- android-5a3c69c045af
14. User
 Timing
 API
15. marks - record individual User Timing events UserTiming API measures - record difference between two marks 91% support + polyfill by Nic Jansma https://github.com/nicjansma/usertiming.js
16. Retrieve the measurements PerformanceTimeline API
17. DEMO UserTiming API
18. • Measures your product’s, not “standard metrics” • Instruments your
 JavaScript codebase not browser codebase • Records timings that can be picked up by both synthetic & RUM • Does not really measure browser painting • Measures specific events in your code rather than user experience • Requires detailed code instrumentation Pros Cons
19. text (w/o a custom font) - inserted into DOM Measure The Browser Steve Souders, “User Timing and Custom Metrics” https://speedcurve.com/blog/user-timing-and-custom-metrics/ Technique by Steve Souders
20. image - inserted into DOM and downloaded Measure The Browser Steve Souders, “User Timing and Custom Metrics” https://speedcurve.com/blog/user-timing-and-custom-metrics/ Technique by Steve Souders
21. Your Code Can Only Measure JavaScript Execution Nolan Lawson, “Accurately measuring layout on the web”
 https://nolanlawson.com/2018/09/25/accurately-measuring-layout-on-the-web/
22. Measure Real Painting “Accurately measuring layout on the web”
 https://nolanlawson.com/2018/09/25/accurately-measuring-layout-on-the-web/ for all marks fired in JS code wait till next paint Technique by Nolan Lawson
23. UX Capture Library
24. Groups multiple marks into “zones” and records
 UserTiming measures between navigationStart and last mark UX Capture
25. Use UXCapture.mark() instead of performance.mark() UX Capture • waits till next paint (can be disabled for specific mark) • records a console.timeStamp() to show marks on timeline
26. DEMO UX Capture
27. • Destination Confirmed • Primary Content Loaded • Primary Action Available • Secondary Action Track same metrics that represent user experience across the site. UX Metrics https://github.com/sergeychernyshev/ux-capture
28. Example: Custom Metrics in SpeedCurve Synthetic • Track as UX Speed Metrics • Correlate UX Speed & KPIs using RUM • Use for analysis with filmstrips and waterfalls to improve User Experience Synthetic
 & RUM
29. Design Speed
30. Thank You! Twitter: @sergeyche Sergey.Chernyshev@gmail.com SergeyChernyshev.com