Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DR2023 Debbie Levvit - Don't Democratize UX Research - And Governance For Those Doing It

DR2023 Debbie Levvit - Don't Democratize UX Research - And Governance For Those Doing It

uxaustralia
PRO

March 15, 2023
Tweet

More Decks by uxaustralia

Other Decks in Design

Transcript

  1. ©2023 Delta CX [email protected] 1
    Don't Democratize UX
    Research –
    And Governance For Those
    Doing It
    Ⓒ 2023. All Rights Reserved. Please credit Debbie Levitt and Delta CX if you are using/reusing/creating derivatives/variations of any of this. Thanks!
    Debbie Levitt

    View Slide

  2. ©2023 Delta CX [email protected] 2
    Connect and join!
    Debbie
    Levitt
    Delta CX
    DeltaCX.link/joinslack
    No sponsorships or affiliate links. Ad revenue donated to charity monthly.
    DeltaCX.link/discord

    View Slide

  3. ©2023 Delta CX [email protected] 6
    6
    Why not
    “democratize?”

    View Slide

  4. ©2023 Delta CX [email protected] 7
    Defining “democratization”
    • It originally meant breaking down silos and making sure that research
    artifacts, reports, and actionable suggestions were “accessible” and shared
    with others.
    Yes, please!
    • Somewhere along the way, “democratization” started meaning that
    research should be done by anybody and everybody. Training and oversight
    are optional. “Anybody can talk to customers!” “Anybody can do what UX
    Researchers do!”
    No, please!
    • To me, “democratization” isn’t when we have Designers and Researchers
    work together on testing. Democratization is when people from outside of
    the CX or UX dept want to do CX/UX tasks, research studies, designs, etc.
    (often without partnering, oversight, or standards).

    View Slide

  5. ©2023 Delta CX [email protected] 9
    Critical thinking questions
    1. What job exists at our company where the quality of work, outputs, and outcomes
    doesn’t matter?
    2. Which other domains will also democratize their work?
    3. What is the ROI of having anybody or everybody do research or design work?
    4. If we’re struggling with a bottleneck, should we stop specialists’ mission critical work
    to try to teach newbies to do mission critical work?
    5. How long will it take for someone to learn some or all of a specialized full-time job?
    6. If the person learning is doing poor work, do we keep that work? Ban that person from
    future CX work? How much risk and waste are we willing to accept?
    7. Aren’t experienced specialists better and more efficient at strategic and tactical CX or
    UX work than a newbie guessing at CX work?
    8. Did we ask CX leaders what they need so that we can best support them?

    View Slide

  6. ©2023 Delta CX [email protected] 11
    Perceived vs actual skills
    People think UX skills are:
    • Talking to people.
    • Making a Customer Journey Map.
    • Putting sticky notes on a board.
    • Using Figma.
    • Laying out screens.
    Skills include:
    • Studies of human behavior and cognitive psychology.
    • Problem finding and solving.
    • Critical thinking.
    • Deductive reasoning and logic.
    • Put your preferences and biases aside to design for user needs.
    • For research:
    Planning the research.
    Choosing the best method(s).
    Planning the correct questions and the tasks we should observe.
    Choosing the right quantity and types of participants.
    Executing sessions with neutrality and a good interviewing style.
    Observing and noticing things others miss. Being a mini-detective.
    Analyzing the data.
    Bringing it together to report on insights, pain points, and
    opportunities.
    Delivering actionable suggestions around pain points, opportunities,
    strategy, and direction.

    View Slide

  7. ©2023 Delta CX [email protected] 12
    “But PMs won’t have empathy!”
    No empathy unless I
    do CX/UX work.
    I’ll also need to
    answer support calls,
    handle Marketing,
    and run QA tests.
    I have empathy.
    I know CX is
    specialized, and they
    don’t want me doing
    their work.
    I’ll be the best
    partner I can be,
    support them, and
    help them get what
    they need.
    I care about
    our customers.
    I’m going to
    watch most of
    the research
    sessions, live
    or recorded.
    I don’t have to
    be the session
    moderator to
    gain first-hand
    knowledge.
    I was a CX/UX
    pro who took
    a PM job so I
    could do the
    CX/UX work I
    was trying to
    do. Now I can
    also do the
    product
    strategy work
    I wanted to
    do.
    ©2023 Delta CX [email protected] 12

    View Slide

  8. ©2023 Delta CX [email protected] 14
    “I Found a Pro-Democratization UX Expert”
    “Anyone can
    cook!”
    “Elitist
    gatekeepers
    who want
    silos!”
    “Anyone with
    a clear, relevant
    question.”
    “Teach and
    then
    influence.”

    View Slide

  9. ©2023 Delta CX [email protected] 16
    16
    Quality over
    speed

    View Slide

  10. ©2023 Delta CX [email protected] 17
    Cost of Poor Quality (COPQ)
    Internal costs:
    • Bad partners/suppliers
    • Delays
    • Failure/root cause analysis
    • Re-designing
    • Rework
    • Re-testing
    • Downtime
    • Lack of flexibility/adaptability
    • Morale
    • Training
    External costs:
    • Complaints (support tickets, etc.)
    • Customers’ trust for you
    • Fewer sales, fewer repeat sales
    • Bad word-of-mouth (ratings,
    tweets, “they suck,” etc.)
    • Environmental costs (physical
    objects)
    • Repairing goods and redoing
    services (customer time &
    frustration even if free)
    • Returns/recalls/exchanges
    • Safety incidents/lawsuits

    View Slide

  11. ©2023 Delta CX [email protected] 19
    Bad evidence is a root cause
    Symptoms During Project
    • No or unclear strategy.
    • No or only business-centric success criteria or KPIs.
    • No/incorrect/customer-peripheric problem statements.
    • Hard to make decisions.
    Symptoms After Release
    • Support utilization.
    • Negative VOC.
    • Lost revenue and customers.
    • Failed A/B test.
    • KPIs/OKRs not met.
    • Blame the Designer for “bad design.”
    Root Cause
    • We didn’t have enough or the right evidence to
    make better decisions, design what users need, or
    solve user problems.

    View Slide

  12. ©2023 Delta CX [email protected] 20
    Unanswered questions

    View Slide

  13. ©2023 Delta CX [email protected] 21
    We solve bottlenecks by hiring

    View Slide

  14. ©2023 Delta CX [email protected] 22
    22
    Governance

    View Slide

  15. ©2023 Delta CX [email protected] 23
    Governance model
    1. Executive support and an enforcement mechanism are
    necessary components.
    2. Root causes and problem statement.
    3. Success criteria.
    4. Accountability for small or large failures.
    5. Training
    6. Priorities
    7. Process
    8. Work and Quality
    9. Costs of Time and Salaries.
    10. Costs of Poor Quality.
    11. Costs of Attrition and Worker Dissatisfaction.

    View Slide

  16. ©2023 Delta CX [email protected] 24
    24
    Think critically
    and take action

    View Slide

  17. ©2023 Delta CX [email protected] 25
    Your company believes in research
    • Surveys
    • Talk to customers
    • Analytics data
    • Focus groups
    • VOC (Voice of the Customer)
    • Customer Support and call centers are data rich.
    • Market research and predictions based on it

    View Slide

  18. ©2023 Delta CX [email protected] 26
    Sample cost-benefit analysis
    Alternative:
    • 20 people & 10% = 2-3 FT CXR
    Efficiency:
    • Experienced specialists better at
    strategy and tactics.
    • Decreases when work is guessed
    at by lightly-trained newbies.
    • Time and cost of delays, lost
    revenue, customer trust, poor user
    outcomes, support utilization.
    UX Research
    “Coach”
    $130K/yr
    6 FT qualified Researchers
    1 Lead, 1 Sr, 4 Juniors
    20 PMs as
    part-time UXers
    $190K/yr
    Mistakes &
    lost revenue
    $200K/yr
    “Democratization” costs $520K/yr
    OR

    View Slide

  19. ©2023 Delta CX [email protected] 28
    Who does CX/UX tasks
    Consider our interview &
    assessment process.
    How many interviews CX/UX went through,
    who assessed them, design/research
    challenges/assignments?
    Qualified for a full-
    time CX/UX job.
    If you wouldn’t qualify for a CX/UX job
    at our company, you shouldn’t be doing
    our work.
    Full-time under CX/UX
    mgmt & standards.
    People choosing CX/UX as their career path.

    View Slide

  20. ©2023 Delta CX [email protected] 30
    Beware “democratization” and “decentralization”
    Why does CX/UX rush to give up power, then complain about
    low maturity, and evangelize how they need power?
    ©2023 Delta CX [email protected] 30

    View Slide

  21. ©2023 Delta CX [email protected] 34
    Thanks for attending!
    • CustomerCentricity.com is in
    progress but has fresh things!
    • Check out my new book,
    “Customers Know You Suck.”
    https://cxcc.to/ckys
    • Free 30 min of coaching for
    each of you. Book “30 Minutes
    Free” at deltacx.link/coaching
    • Questions? [email protected]

    View Slide

  22. ©2023 Delta CX [email protected] 36
    36
    Appendix:
    Governance

    View Slide

  23. ©2023 Delta CX [email protected] 37
    Answer, document, and monitor:
    1. Executive support and an enforcement mechanism are necessary
    components.
    Have we identified the execs or leaders sponsoring this experiment?
    How will we compel compliance with this model when people might want to “break the
    rules?”
    2. Root causes and problem statement.
    What problem are we solving? What do we know about that problem? What’s causing the
    problem?
    What solutions have we considered? Have we analyzed pros and cons of various solutions?
    Why are we diluting CX? Is dilution the best solution?
    ▪ For example, if we are diluting CX because “Product Managers would like to do UX tasks,” is that a good
    enough reason, and are we allowing similar experiments in other domains?
    ▪ If we are diluting CX because CX is a bottleneck, and we’re willing to burn partial non-CX salaries to “get
    the work done,” wouldn’t hiring CX professionals make more sense?

    View Slide

  24. ©2023 Delta CX [email protected] 38
    Answer, document, and monitor:
    3. Success criteria. Before embarking on the democratization experiment,
    we must establish success criteria to know when it’s succeeding or
    failing. Failure signals changing or ending the experiment.
    Our success criteria can’t be any variation of “we did a thing and it got done.” Having
    more people do research tasks might lead to more research tasks being done, but at
    what quality and cost?
    Our goal should be outstanding research that brings us closer to customer-centricity
    and high-quality evidence that will be used in strategies and decisions, not just any
    research.
    The success criteria should be tied to quality and Customer Experience outcomes. It
    should use measurable metrics, not opinions or emotions. “People are enjoying doing
    research” is not a success criterion.

    View Slide

  25. ©2023 Delta CX [email protected] 39
    Answer, document, and monitor:
    3. Success criteria. Before embarking on the democratization experiment,
    we must establish success criteria to know when it’s succeeding or
    failing. Failure signals changing or ending the experiment.
    Our success criteria can’t be any variation of “we did a thing and it got done.” Having
    more people do research tasks might lead to more research tasks being done, but at
    what quality and cost?
    Our goal should be outstanding research that brings us closer to customer-centricity
    and high-quality evidence that will be used in strategies and decisions, not just any
    research.
    The success criteria should be tied to quality and Customer Experience outcomes. It
    should use measurable metrics, not opinions or emotions. “People are enjoying doing
    research” is not a success criterion.

    View Slide

  26. ©2023 Delta CX [email protected] 40
    Answer, document, and monitor:
    4. Accountability for small or large failures. If there are no
    consequences for the people who drove this initiative, then there is
    no incentive to do better and no reason to reduce, change, or stop a
    failing program.
    We said we would hold people responsible or accountable, but what does
    that look like?
    Will we block that decision-maker from leading an experiment in the future?
    Do we reduce their available budget? How about a performance improvement
    plan and coaching?
    Could they be demoted? I have seen companies demote managers and
    leaders; don’t exclude it as a possibility.

    View Slide

  27. ©2023 Delta CX [email protected] 41
    Answer, document, and monitor:
    5. Training. Consider Bloom’s Taxonomy, a
    pedagogical framework visualized as a
    pyramid. In many cases, democratization
    assumes that non-CX staff will jump right
    in at the highest level, which is the ability
    to produce new or original work.
    Does that non-CX worker recall facts and basic
    concepts about CX, the lowest level in this
    model?
    Can they explain CX ideas or concepts (the
    second level) or use CX knowledge in new
    situations (the third level)?
    We cannot expect quality work based on the
    application of extensive knowledge if we are
    not providing that knowledge and taking
    people through these pedagogical stages.

    View Slide

  28. ©2023 Delta CX [email protected] 42
    Answer, document, and monitor:
    5. Training (continued).
    Most companies experimenting with democratization expect a research expert to provide training. Given that the Researcher’s
    full-time job is doing mission critical research tasks, we must create appropriate expectations for the length, breadth, and
    depth of training.
    Some Researchers are great teachers. Some are not. Have we checked which of our Researchers is also great with training
    and pedagogy? Let’s not also dilute education.
    CX research is highly specialized and, on average, takes people years to be good at. We might see low-quality results
    through no fault of the trainer, especially since there may not have been a selection process with standards for who will be
    trying CX work.
    Speaking of which, is there a selection or screening process for non-CX individuals? Can anybody be trained, or do we declare
    some people not a match to CX work?
    Some students will never be good at research tasks, no matter how much training they get. There will be people who just
    don’t get it.
    How much training will there be? What types of training? Will learners have time to practice on non-urgent projects?
    When does our Researcher have time to shift into Instructional Designer mode and create a training program? Are we sending
    our non-Researchers to an outside training course or bringing in a corporate trainer? How did we investigate and vet the
    quality of that course or that trainer?
    Will you utilize formative and summative assessments of the students and the trainer to measure process and effectiveness?

    View Slide

  29. ©2023 Delta CX [email protected] 43
    Answer, document, and monitor:
    6. Priorities. Have we codified which work should go to the professional
    Researchers and which work should go to the non-Researchers? How will we
    prioritize the incoming work, and match work to practitioners?
    What are the priorities for the people being asked to train others? Which is more important:
    the work they currently have on their plate or the training and work reviews for those who are
    learning? Do not say, “They can do both.” People with packed schedules need clear priorities.
    What are the priorities for the people trying to learn or do research tasks? Which is more
    important: the work they currently have on their plate or the training and research work? Do
    not say, “They can do both.”
    7. Process. A good research process includes planning, recruiting participants,
    executing the sessions, analyzing the data, synthesizing findings, and arriving at
    actionable insights.
    Will non-Researchers be required to execute a correct research process? If not, why not?

    View Slide

  30. ©2023 Delta CX [email protected] 44
    Answer, document, and monitor:
    8. Work and Quality. Will non-Researchers doing research tasks be required to meet ISO or other
    standards we have documented? If not, why not?
    If we find that someone’s work doesn’t meet our quality standards, what do we do about that? Do we ban them
    from future research? Do we invest more time and money in trying to train them further?
    Do we include the bad research or deliver it to a client? Do we scrap the research and pretend it never
    happened?
    Some well-paid trainers say that research work done by non-Researchers does not need to be checked. We
    can let it go and assume it’s “good enough.” If we are not checking this work, that should be documented as
    high risk.
    All work that is in progress and completed should be documented and archived. This includes documentation
    of every step of our research process. We should archive our session recordings, research plans, and artifacts
    relating to how this data was analyzed.
    9. Costs of Time and Salaries. Calculate the time and salary a Researcher spends training,
    overseeing, coaching, reviewing work, correcting work, etc. Calculate the time and salary a non-
    Researcher spends learning and doing research.
    Did work have to be redone by someone with higher skill? That took time and salary.

    View Slide

  31. ©2023 Delta CX [email protected] 45
    Answer, document, and monitor:
    10. Costs of Poor Quality. Are we monitoring the projects using non-Researchers for research? Have we
    checked if research quality set a later stage of the project up for mistakes or failure?
    For example, was usability testing highly flawed, invalid, or didn’t deliver the information Designers needed to improve their
    designs? Did something that research “told” us turn out to not be accurate?
    Did we release something to the public or deliver it to a client, but it’s partially or wholly failing? What did that cost us in
    reputation, stock price, customer trust, Customer Support utilization, negative word of mouth, or customers downgrading or
    leaving?
    Did we make a big enough mistake where we had to roll back a software release, undo a price increase, or negate something
    we delivered to the public? What are the costs of that mistake and the efforts to fix or undo it?
    11. Costs of Attrition and Worker Dissatisfaction. Did anybody quit our company partially or wholly due to
    democratization? It would be a serious loss if that knowledge and capability walked out the door.
    Researchers might not want to stay in a workplace where anybody can do their job regardless of proficiency. Non-
    Researchers might not like more tasks dumped on them, especially if they already have no spare time.
    Your workers might stay, but are they happy? If you opened a job in their domain, would they recommend that their friends
    work here? Are they telling their network that this is a great place to work?
    We must monitor all of these. And with some effort, we could calculate the time and money spent on most, if not all, of these.
    We should be able to quantify what this experiment costs, and compare that to the salaries that we could have paid
    professionals to do this work.

    View Slide