Difficult usability testing participants

A look at difficult participants through the lease of Mr men and little miss!

Transcript

1. @laura_yarrow Difficult research participants …and (non-sinister) ways of dealing with them
2. @laura_yarrow Difficult user testing participants 2 ★ Research in labs/in office ★ In-house or agency ★ Recruiting participants ★ Organising the day ★ Managing a client and observers This covers:
3. @laura_yarrow Difficult user testing participants 3 Sometimes, no matter what you do, tricky participants just happen.
4. @laura_yarrow Difficult user testing participants 4 Disclaimer: Most user testing participants are reacting to an unnatural situation
5. @laura_yarrow Difficult user testing participants 5 The many types of participant to deal with…
6. @laura_yarrow Difficult user testing participants 6 The Chatterbox ★ Spend all the session off topic ★ Tell you their life story - use the session as a social event ★ Lengthy discussions about the task ★ Don’t take it seriously Tips: ✓ Put a large clock on the desk ✓ Time-boxed script ✓ Don’t be afraid to interrupt ✓ Use closed questions at the end to get to the point MR CHATTERBOX
7. @laura_yarrow Difficult user testing participants 7 2. The Mute ★ Nervous, detached, despondent, or naturally quiet ★ Recent trauma or bad situation ★ Won’t talk through what they’re doing, afraid we’re testing them ✓ Be kind ✓ Be ready with open ended questions ✓ Have an introduction at the start that states they’re not being tested MR QUIET Tips:
8. @laura_yarrow Difficult user testing participants 8 3. The Pleasers ★ Wants to tell you what you want to hear ★ Won’t criticise anything, even if they struggle ★ If asked to rate anything, will give 10/10 ★ Will rush through everything like a race ✓ Screen for their sentiment towards brands/clients ✓ Emphasise your neutrality ✓ Play devils advocate - say you have to bring back a balanced set of feedback MR PERFECT Tips:
9. @laura_yarrow Difficult user testing participants 9 4. The Nosey Ones ★ Grill you about the client or organisation ★ Find it hard to focus on the tasks and research ★ May start using their phone/your tech to look up details! ✓ Again, reiterate your independence ✓ Let them know you can tell them at the end ✓ Don’t name anyone ✓ Test a range of products/sites/apps to maintain neutrality MR NOSEY Tips:
10. @laura_yarrow Difficult user testing participants 10 5. The Cheaters ★ Lied on the screener! ★ Trying to make a profession out of being a research participant ★ Will skew your research findings ★ Say they use the tech or devices required but can’t in the session ✓ Compare previous answers to screeners ✓ Keep a blacklist - take note of names that respond regularly ✓ Always recruit a backup LITTLE MISS NAUGHTY Tips:
11. @laura_yarrow Difficult user testing participants 11 6. The Undercover Agent ★ Refuses to be recorded or observed ★ Highly suspicious, very wary of recording devices ★ May have had a past bad experience ✓ Ask them in the screener if they are happy to be recorded in the session ✓ Give them a copy of the consent form ✓ Let them know exactly what you do with the recordings ✓ In the worst case, fall back to audio recording, or good note-taking! MR WORRY Tips:
12. @laura_yarrow Difficult user testing participants 12 7. The Free Spirit ✓ Time-box your script and tasks ✓ State “I want to spend 5 minutes talking about…” ✓ Keep a clock on the desk and keep referring to it ✓ Send instructions before hand about thinking out loud so they can practice LITTLE MISS SCATTERBRAIN ★ Regularly goes off course, distracted ★ Opening new tabs in browsers, not sticking to tasks ★ Incoherently narrates, can’t think out loud ★ Will show you photos on their phone of their cat/grandkids/plant ★ Possibly eating a MacDonalds lunch throughout Tips:
13. @laura_yarrow Difficult user testing participants 13 8. The Reluctant Participant ★ Treat it like an interview ★ Won’t touch the tech or engage ★ Think it’s a quiz and try and guess the answers to tasks without doing them ✓ Set expectations if you want them to use some devices ✓ “Can you show me how you would do this?” ✓ “Without telling me…” ✓ Gesture at the devices you want them to use… LITTLE MISS STUBBORN Tips:
14. @laura_yarrow Difficult user testing participants 14 9. The Confused ★ Not sure why they’re here / never done this before ★ Doesn’t normally use the tech you’re expecting them to use ★ Thinks they can bring their friend/child/dog… MR MUDDLE Tips: ✓ Send a guide before hand ✓ Brief them in at the start (what/why), and have they done this before? ✓ Only one question at a time ✓ Screen for devices they’re familiar with / technical ability
15. @laura_yarrow Difficult user testing participants 15 10. The Inappropriate Person ★ Wants to talk about you more than the tasks ★ Makes inappropriate comments ★ Poor hygiene etc ★ Inadvertently in-appropriate ✓ Don’t go alone, take a colleague who’s always watching ✓ End the test if you feel uncomfortable ✓ You can position them where you want! LITTLE MISS SCARY Tips:
16. @laura_yarrow Difficult user testing participants 16 11. The No Show ★ Doesn’t let you know ★ Doesn’t answer the phone ★ May turn up really late and try and slot themselves in ★ Will suggest a date in 4 weeks when they have a slot free… ✓ Blacklist ✓ Capture their phone number and text the night before ✓ Have a backup participant, or “floater” ✓ Allow breaks between each session where you could squeeze them in MR NOBODY Tips:
17. @laura_yarrow Difficult user testing participants 17 Summary ✓ Send out information before ✓ Have a time-boxed script to follow ✓ Text the night before ✓ Have a backup participant ✓ Screen properly (device, sentiment, job…) ✓ Brief the participant at the start ✓ Hide the tech ✓ Tell them what you do with recordings ✓ Allow for 5 minutes of settling in at the start ✓ Take a colleague MR CLEVER
18. @laura_yarrow Difficult user testing participants 18 Thanks, any questions? laurayarrow@gmail.com Email: Twitter: @laura_yarrow All images courtesy of www.mrmen.com