1000 JavaScript errors are as good as None

Does your site’s webdev console show thousands of errors on a user’s browser but none on yours? Do you receive complaints from users that they’re unable to complete a purchase and you can’t reproduce it on your end?

When studying JavaScript errors, we can easily get lost in a very large number of errors that all seem to say the same thing. How do we tell if different errors really refer to the same root cause? How do we tell which errors are the most important to address? If we had to pick just one error to fix, how do we tell which one has the most impact on our users and business?

In this talk, we look at ways to de-duplicate JavaScript errors across browsers and devices, and then how to identify which ones affect our users the most.

Do some errors block the page from rendering? Do other errors cause increased user frustration? Is there an error that prevents users from finishing a checkout flow?

Presented at: iJS Munich

Transcript

1. 1000 JavaScript Errors are as good as None! Identifying the most important JavaScript Errors
2. Philip Tellis Principal RUM Distiller @ Akamai • Analyses real user performance data from mPulse • Author of the OpenSource boomerang RUM library twitter:@bluesmoon ⦿ github:@bluesmoon speakerdeck:@bluesmoon
3. Needles in a JavaScript Haystack(Trace) ★ Errors? What errors? ★ Too Much Information! ★ Ugh! Help me fix them 🙏
4. Hi all, Customer complains that we are blocking the “Fortsetzung ohne Akzeptanz” button on the website… they see this error
5. Developer: I can click on “Fortsetzung ohne Akzeptanz” and it works for me. I don’t see any error in the stack.
6. WFM!
7. User Machines are Strange Beasts! They are a mystery to us…
8. Boomerang! • A JavaScript library to mainly collect performance data from the browser. • It also collects information about JavaScript errors. ★ onerror global event handler ★ XMLHttpRequest responses that were not successful ★ Any calls to window.console.error ★ Runtime errors that happen during a callback for addEventListener, setTimeout and setInterval ★ And more! Note: Make sure cross-origin scripts have the crossorigin="anonymous" attribute and Access-Control-Allow-Origin header set. Code: https://github.com/akamai/boomerang Docs: https://techdocs.akamai.com/mpulse-boomerang/docs/errors
9. 25-120 million JavaScript errors every Day!
10. e=null; e.length Uncaught TypeError: Cannot read properties of null (reading 'length') TypeError: null is not an object (evaluating 'e.length') Unable to get value of the property 'length': object is null or undefined Uncaught TypeError: e is null
11. TypeError: A conexão de rede foi perdida TypeError: De netwerkverbinding is verbroken TypeError: Die Netzwerkverbindung wurde unterbrochen TypeError: La conexión de red se ha perdido TypeError: La conexión de red se perdió TypeError: La connexion réseau a été perdue TypeError: Mistet nettverksforbindelsen TypeError: Netværksforbindelsen gik tabt Grüß Gott Fehler, Salut erreur, नमस्ते गलती TypeError: Nätverksanslutningen förlorades TypeError: Połączenie sieciowe zostało utracone TypeError: Pripojenie k sieti bolo stratené TypeError: The network connection was lost TypeError: Сетевое соединение потеряно TypeError: ネットワーク接続が切れました。 TypeError: 網路連線中斷。 TypeError: 네트워크 연결이 유실되었습니다
12. TypeError: Abgebrochen TypeError: Kumottu TypeError: annullato TypeError: annulleret TypeError: annulé TypeError: anulat TypeError: avbruten TypeError: avbrutt TypeError: cancelado Kumottu, Abgebrochen, キャンセルしました TypeError: cancelled TypeError: geannuleerd TypeError: vazgeçildi TypeError: visszavonva TypeError: отменено TypeError: ถูกยกเลิก TypeError: キャンセルしました TypeError: 中止 TypeError: 已取消
13. TypeError: A server with the specified hostname could not be found TypeError: Aucun serveur ayant le nom d’hôte précisé n’a pu être trouvé TypeError: Aucun serveur ayant le nom d’hôte précisé n’a été détecté TypeError: Es wurde kein Server mit dem angegebenen Hostnamen gefunden TypeError: The Internet connection appears to be offline TypeError: The internet connection appears to be offline Even different dialects
14. e=null; e… Cannot read properties of undefined (reading 'appendChild') Cannot read properties of undefined (reading 'childNodes') Cannot read properties of null (reading 'classList') Cannot read properties of null (reading 'innerHTML') Cannot read properties of null (reading '__vue__') Cannot read properties of null (reading 'add') Cannot read properties of null (reading 'addEventListener')
15. Script Error [object Object] zoid destroyed all components Error: Syntax error ReferenceError: 'google' is undefined window is not an object (empty string) Plugin errors 🙀
16. DS 101: clean the data • Different browsers reports the same error differently • Errors are reported as text messages rather than numeric codes • A certain browser also localizes messages based on browser locale. • For any given issue, there will be many errors reported – one for each side-effect of that error. • And then some errors are just useless.
17. Start in the Browser • Browsers are consistent within the same page view • We can pre-aggregate messages into groups before sending them to the server. Down to 10-30 million errors a Day 🎉
18. Step 2: Deal with the L10N • Localized strings are tackled with a large mapping table. • Normalizes all strings to their en-US equivalent. • In some cases there are immutable parts, like variable identifiers that must remain unchanged, but change position. Down to 5-15 million errors a Day
19. Step 3: Common Error Names • List of 300 regular expressions to munge errors down to a common format. • Some of the regexes are chained. Down to 2-8 million errors a Day
20. Step 4: Uniqueness • We can now look at unique errors for analysis, and what we see is surprising… Between 300-1700 unique errors a Day 🎉
21. Copyright Randall Munroe https://xkcd.com/2343/
22. Clustering • We have several parameters to cluster on. • Filename & line number are useful. • In most cases, the error text is the best indicator of its cluster. Down to 20-100 groups
23. Methods of Clustering • Density-based spatial clustering of applications with noise (DBSCAN) • K-Nearest-Neighbors (KNN) • K-Means • K-Medoids You’ll need to figure out a distance function for the dataset Martin Ester, Hans-Peter Kriegel, Jiirg Sander, Xiaowei Xu A density based approach for discovering Clusters in Large Spatial Databases with Noise
24. Error: Expected ')' Error: Expected ':' Error: Expected ';' Error: Expected '}' Error: Expected ']' Error: Expected '(' Error: Expected identifier Error: Expected hexadecimal digit Error: Expected identifier, string or number Example Clusters Cannot read properties of null (reading '1') Cannot read properties of null (reading '0') Cannot read properties of null (reading 'add') Cannot read properties of null (reading 'blur') Cannot read properties of null (reading '7dvn') Cannot read properties of null (reading 'width') Cannot read properties of null (reading 'click') Cannot read properties of null (reading 'style') Cannot read properties of null (reading 'value') Cannot read properties of null (reading 'remove') Cannot read properties of null (reading '__vue__') Cannot read properties of null (reading 'getItem') Cannot read properties of null (reading 'setItem') Cannot read properties of null (reading 'nodeType') Cannot read properties of null (reading 'errorCode') Cannot read properties of null (reading 'startsWith') Cannot read properties of null (reading 'postMessage') Cannot read properties of null (reading 'removeChild')
25. Error Impact Identifying which errors have an impact
26. Which one do I fix first? • Let’s look at errors that impact the user journey. • Focus on errors that affect a user’s ability to convert. • Can we tell if certain errors frustrate the user? • What’s the performance impact of errors?
27. ↻ Reload Rate • Do users hit the reload button when this error occurs? • Use the type property from the PerformanceNavigationTiming object. • Make sure sample size is large enough, i.e. 100/500 reloads is more reliable than 3/6 reloads even though it’s 20% v/s 50%
28. 😠 Rage Click Rate • Does the existence of a particular error correspond to a high number of Rage Clicks on the page with the error?
29. Rage Clicks occur when users rapid-fire click (or tap) on your site or app. Rage clicking is the digital equivalent of cursing to release frustration. https://www.fullstory.com/resources/guide-to-understanding-frustrating-user-experiences-online/ https://www.psychologytoday.com/us/blog/hide-and-seek/201205/hell-yes-the-7-best-reasons-swearing Rage Clicks
30. 😠 Rage Click Rate • Does the existence of a particular error correspond to a high number of Rage Clicks on the page with the error? • Points to user frustration
31. Truncated User Journey • Some errors cause a user session to terminate immediately. • Others seem to allow the user to proceed. • It’s also instructive to split this report by converted and/or “might convert” status.
32. Errors affecting Performance • Some errors make page load time worse. • In this case inability to access localStorage probably requires extra network calls. Access denied for ‘localStorage’
33. Errors affecting Performance • Some errors make page load time better. • In some cases, it causes page load to fail resulting in partial or no-content. There was an error setting cookie .* Please check domain and path
34. If an error blocks content and makes the page faster, without reloads or truncated sessions, then is that content really needed?
35. Debugging Aids Information to identify root causes
36. How do I fix them? • When did it start happening? related to a new product release, browser release, etc. • Where does it happen? Browser, OS, specific page, Geo region, etc. • What are the conditions when it happens? Specific scripts/css loaded/not loaded, load time, http/h2, etc. • What was the user journey like when it happens? Previous page, SPA lifetime, Session Length
37. Errors over Time • Look at total error count as well as average errors per page for a given error. • Check if new spikes or fixes correlate with a new product release or a new browser release (we generally annotate our charts with product releases). • For existing errors, look at trends in the average error count.
38. Where does the error happen? OS Browser Device The operation is insecure Country Page
39. Conditions corresponding to Errors Some errors only show up when particular JavaScript files fail to load… for example, due to an Ad-Blocker or high packet loss
40. Some errors show up when a particular JavaScript file loads up more than once!
41. I had the same error, it seems to have been caused by loading the PayPal script more than once. I fixed it by loading the script just once in the head, then loading the two buttons where they should be. zoid destroyed all components “ RebeccaRey on github issues for paypal
42. Some errors show up when the page is proxied through Google Translate
43. CSRF token expired. Some errors often show up when particular resources were served from cache.
44. Some errors show up when the client device is in low power mode (battery < 10%).
45. Effects of User Journey • Data corruption in cookies or localStorage might only show up after a user browses through a few pages. • Memory leaks in Single Page Apps may only manifest as problems after several navigations. • Cross-domain cookie issues could show up if the user had to pass through a different subdomain through their journey. 🍪 🥠
46. Dealing with Script Error
47. What’s the Problem? Script error 51,169,268 [container] is not a valid DOM Element 2,845,511 [object Object] 2,553,637 ReferenceError: 'xxxxxx' is undefined 1,952,751 Error: s:0 rs:4 1,445,856 Attempting to change access mechanism for an unconfigurable property 1,083,304 TypeError: [code] is not a function 809,376 TypeError: undefined is not an object 789,812 81% No information in Script Error
48. Why does this happen? • Cross-origin script files that may be credentialed. • The browser doesn’t know if error information may be sensitive or not, so it hides it by default. • For example, an error that looks like this in the web dev console…
49. Looks like this in an onerror handler… • No real error message, no filename, line number, stack, nothing. • You only get a timestamp!
50. Two options to fix it 1. If you have control over the page & script domain, do the following: a. Set the crossorigin="anonymous" attribute on the <script> element. This tells the browser not to send cookies, and that this request is non-credentialed. b. Set the Access-Control-Allow-Origin HTTP response header on the script to either * or the domain of your host page.
51. Two options to fix it 2. If you only have control over the page, do the following: ◦ Wrap all your calls to 3rd party libraries in a try/catch block: try { libraryCall(); } catch(e) { // e contains detailed information about the error }
52. ★ Collect user errors ★ Normalize & De-duplicate ★ Cluster similar errors ★ Measure User Impact ★ Handle Script Error Summary
53. Thank you!
54. • boomerang error plugin: Source on github • Docs at Akamai • null has no properties – MDN docs • zoid destroyed all components – github issue • A density based approach for discovering Clusters in Large Spatial Databases with Noise – Ester M, Kriegel H-P, Sander J, Xu X • PerformanceNavigationTiming.type – MDN docs • Rage Clicking – Fullstory • Web Performance with Android's Battery Saver Mode – Goel U, Ludin S, Steiner M • Side effects of boomerang’s JS Error tracking – Nic Jansma References