Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using Analytics to Improve UX

Avatar for Paul McMahon Paul McMahon
May 01, 2014
3.4k

Using Analytics to Improve UX

Avatar for Paul McMahon

Paul McMahon

May 01, 2014
Tweet

Transcript

  1. New Relic • Tells us if our site is up

    • Identifies slow response times
  2. Sentry • Monitors application errors such as crashes • Less

    than 1% of users will report errors • Record user id if possible so you can follow up with the user after
  3. Crazy Egg • Creates a heatmap of where users click

    on your page • Can be useful in identifying things that should be clickable, but aren't
  4. Kiss Metrics • Record arbitrary events like “View Event Page”,

    “Start Registration”, and “Complete Registration” • Set arbitrary properties like “Subscription Plan” on events/users
  5. Funnel Reports 
 (Kiss Metrics) • Track how users move

    through a series of steps towards a goal • Identify where people drop out, which gives you of ideas where you can focus UX effort • Track your changes over time
  6. Measuring Improvements • Using data, we can come up with

    hypothesizes for UX improvements • Let’s imagine we are trying to improve Doorkeeper’s event page Existing Experiment
  7. A/B Testing • Put each visitor into a control or

    experimental group randomly • Show existing version to control group,
  8. The larger your sample size, the smaller the improvement you

    can measure • If we have 2000 people visit our event page, one version needs to outperform the other by 25% to be statistically significant • With 20000 people, it drops to 8% • Hypothesize appropriate sample size before running experiment
  9. Measure conversion rate to the goal • Make sure you

    measure the conversion rate of what is actually your goal. • If you have a multi-step funnel, optimizations can increase the first step, but still lead to a lower overall conversion.
  10. Read More • A/B testing is powerful but easy to

    abuse if you don’t understand the statistics behind it • See Jason Cohen’s Business of Software Conference talk “Why data will deceive you”.