DR2023 Debbie Levvit - Don't Democratize UX Research - And Governance For Those Doing It

©2023 Delta CX [email protected] 1
Don't Democratize UX
Research –
And Governance For Those
Doing It
Ⓒ 2023. All Rights Reserved. Please credit Debbie Levitt and Delta CX if you are using/reusing/creating derivatives/variations of any of this. Thanks!
Debbie Levitt

View Slide

Connect and join!
Debbie
Levitt
Delta CX
DeltaCX.link/joinslack
No sponsorships or affiliate links. Ad revenue donated to charity monthly.
DeltaCX.link/discord

View Slide

6
Why not
“democratize?”

View Slide

Defining “democratization”
• It originally meant breaking down silos and making sure that research
artifacts, reports, and actionable suggestions were “accessible” and shared
with others.
Yes, please!
• Somewhere along the way, “democratization” started meaning that
research should be done by anybody and everybody. Training and oversight
are optional. “Anybody can talk to customers!” “Anybody can do what UX
Researchers do!”
No, please!
• To me, “democratization” isn’t when we have Designers and Researchers
work together on testing. Democratization is when people from outside of
the CX or UX dept want to do CX/UX tasks, research studies, designs, etc.
(often without partnering, oversight, or standards).

View Slide

Critical thinking questions
1. What job exists at our company where the quality of work, outputs, and outcomes
doesn’t matter?
2. Which other domains will also democratize their work?
3. What is the ROI of having anybody or everybody do research or design work?
4. If we’re struggling with a bottleneck, should we stop specialists’ mission critical work
to try to teach newbies to do mission critical work?
5. How long will it take for someone to learn some or all of a specialized full-time job?
6. If the person learning is doing poor work, do we keep that work? Ban that person from
future CX work? How much risk and waste are we willing to accept?
7. Aren’t experienced specialists better and more efficient at strategic and tactical CX or
UX work than a newbie guessing at CX work?
8. Did we ask CX leaders what they need so that we can best support them?

View Slide

Perceived vs actual skills
People think UX skills are:
• Talking to people.
• Making a Customer Journey Map.
• Putting sticky notes on a board.
• Using Figma.
• Laying out screens.
Skills include:
• Studies of human behavior and cognitive psychology.
• Problem finding and solving.
• Critical thinking.
• Deductive reasoning and logic.
• Put your preferences and biases aside to design for user needs.
• For research:
Planning the research.
Choosing the best method(s).
Planning the correct questions and the tasks we should observe.
Choosing the right quantity and types of participants.
Executing sessions with neutrality and a good interviewing style.
Observing and noticing things others miss. Being a mini-detective.
Analyzing the data.
Bringing it together to report on insights, pain points, and
opportunities.
Delivering actionable suggestions around pain points, opportunities,
strategy, and direction.

View Slide

“But PMs won’t have empathy!”
No empathy unless I
do CX/UX work.
I’ll also need to
answer support calls,
handle Marketing,
and run QA tests.
I have empathy.
I know CX is
specialized, and they
don’t want me doing
their work.
I’ll be the best
partner I can be,
support them, and
help them get what
they need.
I care about
our customers.
I’m going to
watch most of
the research
sessions, live
or recorded.
I don’t have to
be the session
moderator to
gain first-hand
knowledge.
I was a CX/UX
pro who took
a PM job so I
could do the
CX/UX work I
was trying to
do. Now I can
also do the
product
strategy work
I wanted to
do.

View Slide

“I Found a Pro-Democratization UX Expert”
“Anyone can
cook!”
“Elitist
gatekeepers
who want
silos!”
“Anyone with
a clear, relevant
question.”
“Teach and
then
influence.”

View Slide

16
Quality over
speed

View Slide

Cost of Poor Quality (COPQ)
Internal costs:
• Bad partners/suppliers
• Delays
• Failure/root cause analysis
• Re-designing
• Rework
• Re-testing
• Downtime
• Lack of flexibility/adaptability
• Morale
• Training
External costs:
• Complaints (support tickets, etc.)
• Customers’ trust for you
• Fewer sales, fewer repeat sales
• Bad word-of-mouth (ratings,
tweets, “they suck,” etc.)
• Environmental costs (physical
objects)
• Repairing goods and redoing
services (customer time &
frustration even if free)
• Returns/recalls/exchanges
• Safety incidents/lawsuits

View Slide

Bad evidence is a root cause
Symptoms During Project
• No or unclear strategy.
• No or only business-centric success criteria or KPIs.
• No/incorrect/customer-peripheric problem statements.
• Hard to make decisions.
Symptoms After Release
• Support utilization.
• Negative VOC.
• Lost revenue and customers.
• Failed A/B test.
• KPIs/OKRs not met.
• Blame the Designer for “bad design.”
Root Cause
• We didn’t have enough or the right evidence to
make better decisions, design what users need, or
solve user problems.

View Slide

Unanswered questions

View Slide

We solve bottlenecks by hiring

View Slide

22
Governance

View Slide

Governance model
1. Executive support and an enforcement mechanism are
necessary components.
2. Root causes and problem statement.
3. Success criteria.
4. Accountability for small or large failures.
5. Training
6. Priorities
7. Process
8. Work and Quality
9. Costs of Time and Salaries.
10. Costs of Poor Quality.
11. Costs of Attrition and Worker Dissatisfaction.

View Slide

24
Think critically
and take action

View Slide

Your company believes in research
• Surveys
• Talk to customers
• Analytics data
• Focus groups
• VOC (Voice of the Customer)
• Customer Support and call centers are data rich.
• Market research and predictions based on it

View Slide

Sample cost-benefit analysis
Alternative:
• 20 people & 10% = 2-3 FT CXR
Efficiency:
• Experienced specialists better at
strategy and tactics.
• Decreases when work is guessed
at by lightly-trained newbies.
• Time and cost of delays, lost
revenue, customer trust, poor user
outcomes, support utilization.
UX Research
“Coach”
$130K/yr
6 FT qualified Researchers
1 Lead, 1 Sr, 4 Juniors
20 PMs as
part-time UXers
$190K/yr
Mistakes &
lost revenue
$200K/yr
“Democratization” costs $520K/yr
OR

View Slide

Who does CX/UX tasks
Consider our interview &
assessment process.
How many interviews CX/UX went through,
who assessed them, design/research
challenges/assignments?
Qualified for a full-
time CX/UX job.
If you wouldn’t qualify for a CX/UX job
at our company, you shouldn’t be doing
our work.
Full-time under CX/UX
mgmt & standards.
People choosing CX/UX as their career path.

View Slide

Beware “democratization” and “decentralization”
Why does CX/UX rush to give up power, then complain about
low maturity, and evangelize how they need power?

View Slide

Thanks for attending!
• CustomerCentricity.com is in
progress but has fresh things!
• Check out my new book,
“Customers Know You Suck.”
https://cxcc.to/ckys
• Free 30 min of coaching for
each of you. Book “30 Minutes
Free” at deltacx.link/coaching
• Questions? [email protected]

View Slide

36
Appendix:
Governance

View Slide

Answer, document, and monitor:
1. Executive support and an enforcement mechanism are necessary
components.
Have we identified the execs or leaders sponsoring this experiment?
How will we compel compliance with this model when people might want to “break the
rules?”
2. Root causes and problem statement.
What problem are we solving? What do we know about that problem? What’s causing the
problem?
What solutions have we considered? Have we analyzed pros and cons of various solutions?
Why are we diluting CX? Is dilution the best solution?
▪ For example, if we are diluting CX because “Product Managers would like to do UX tasks,” is that a good
enough reason, and are we allowing similar experiments in other domains?
▪ If we are diluting CX because CX is a bottleneck, and we’re willing to burn partial non-CX salaries to “get
the work done,” wouldn’t hiring CX professionals make more sense?

View Slide

3. Success criteria. Before embarking on the democratization experiment,
we must establish success criteria to know when it’s succeeding or
failing. Failure signals changing or ending the experiment.
Our success criteria can’t be any variation of “we did a thing and it got done.” Having
more people do research tasks might lead to more research tasks being done, but at
what quality and cost?
Our goal should be outstanding research that brings us closer to customer-centricity
and high-quality evidence that will be used in strategies and decisions, not just any
research.
The success criteria should be tied to quality and Customer Experience outcomes. It
should use measurable metrics, not opinions or emotions. “People are enjoying doing
research” is not a success criterion.

View Slide

4. Accountability for small or large failures. If there are no
consequences for the people who drove this initiative, then there is
no incentive to do better and no reason to reduce, change, or stop a
failing program.
We said we would hold people responsible or accountable, but what does
that look like?
Will we block that decision-maker from leading an experiment in the future?
Do we reduce their available budget? How about a performance improvement
plan and coaching?
Could they be demoted? I have seen companies demote managers and
leaders; don’t exclude it as a possibility.

View Slide

5. Training. Consider Bloom’s Taxonomy, a
pedagogical framework visualized as a
pyramid. In many cases, democratization
assumes that non-CX staff will jump right
in at the highest level, which is the ability
to produce new or original work.
Does that non-CX worker recall facts and basic
concepts about CX, the lowest level in this
model?
Can they explain CX ideas or concepts (the
second level) or use CX knowledge in new
situations (the third level)?
We cannot expect quality work based on the
application of extensive knowledge if we are
not providing that knowledge and taking
people through these pedagogical stages.

View Slide

5. Training (continued).
Most companies experimenting with democratization expect a research expert to provide training. Given that the Researcher’s
full-time job is doing mission critical research tasks, we must create appropriate expectations for the length, breadth, and
depth of training.
Some Researchers are great teachers. Some are not. Have we checked which of our Researchers is also great with training
and pedagogy? Let’s not also dilute education.
CX research is highly specialized and, on average, takes people years to be good at. We might see low-quality results
through no fault of the trainer, especially since there may not have been a selection process with standards for who will be
trying CX work.
Speaking of which, is there a selection or screening process for non-CX individuals? Can anybody be trained, or do we declare
some people not a match to CX work?
Some students will never be good at research tasks, no matter how much training they get. There will be people who just
don’t get it.
How much training will there be? What types of training? Will learners have time to practice on non-urgent projects?
When does our Researcher have time to shift into Instructional Designer mode and create a training program? Are we sending
our non-Researchers to an outside training course or bringing in a corporate trainer? How did we investigate and vet the
quality of that course or that trainer?
Will you utilize formative and summative assessments of the students and the trainer to measure process and effectiveness?

View Slide

6. Priorities. Have we codified which work should go to the professional
Researchers and which work should go to the non-Researchers? How will we
prioritize the incoming work, and match work to practitioners?
What are the priorities for the people being asked to train others? Which is more important:
the work they currently have on their plate or the training and work reviews for those who are
learning? Do not say, “They can do both.” People with packed schedules need clear priorities.
What are the priorities for the people trying to learn or do research tasks? Which is more
important: the work they currently have on their plate or the training and research work? Do
not say, “They can do both.”
7. Process. A good research process includes planning, recruiting participants,
executing the sessions, analyzing the data, synthesizing findings, and arriving at
actionable insights.
Will non-Researchers be required to execute a correct research process? If not, why not?

View Slide

8. Work and Quality. Will non-Researchers doing research tasks be required to meet ISO or other
standards we have documented? If not, why not?
If we find that someone’s work doesn’t meet our quality standards, what do we do about that? Do we ban them
from future research? Do we invest more time and money in trying to train them further?
Do we include the bad research or deliver it to a client? Do we scrap the research and pretend it never
happened?
Some well-paid trainers say that research work done by non-Researchers does not need to be checked. We
can let it go and assume it’s “good enough.” If we are not checking this work, that should be documented as
high risk.
All work that is in progress and completed should be documented and archived. This includes documentation
of every step of our research process. We should archive our session recordings, research plans, and artifacts
relating to how this data was analyzed.
9. Costs of Time and Salaries. Calculate the time and salary a Researcher spends training,
overseeing, coaching, reviewing work, correcting work, etc. Calculate the time and salary a non-
Researcher spends learning and doing research.
Did work have to be redone by someone with higher skill? That took time and salary.

View Slide

10. Costs of Poor Quality. Are we monitoring the projects using non-Researchers for research? Have we
checked if research quality set a later stage of the project up for mistakes or failure?
For example, was usability testing highly flawed, invalid, or didn’t deliver the information Designers needed to improve their
designs? Did something that research “told” us turn out to not be accurate?
Did we release something to the public or deliver it to a client, but it’s partially or wholly failing? What did that cost us in
reputation, stock price, customer trust, Customer Support utilization, negative word of mouth, or customers downgrading or
leaving?
Did we make a big enough mistake where we had to roll back a software release, undo a price increase, or negate something
we delivered to the public? What are the costs of that mistake and the efforts to fix or undo it?
11. Costs of Attrition and Worker Dissatisfaction. Did anybody quit our company partially or wholly due to
democratization? It would be a serious loss if that knowledge and capability walked out the door.
Researchers might not want to stay in a workplace where anybody can do their job regardless of proficiency. Non-
Researchers might not like more tasks dumped on them, especially if they already have no spare time.
Your workers might stay, but are they happy? If you opened a job in their domain, would they recommend that their friends
work here? Are they telling their network that this is a great place to work?
We must monitor all of these. And with some effort, we could calculate the time and money spent on most, if not all, of these.
We should be able to quantify what this experiment costs, and compare that to the salaries that we could have paid
professionals to do this work.

View Slide

DR2023 Debbie Levvit - Don't Democratize UX Research - And Governance For Those Doing It

DR2023 Debbie Levvit - Don't Democratize UX Research - And Governance For Those Doing It

uxaustralia
PRO

More Decks by uxaustralia

Other Decks in Design

Featured

Transcript

DR2023 Debbie Levvit - Don't Democratize UX Research - And Governance For Those Doing It

DR2023 Debbie Levvit - Don't Democratize UX Research - And Governance For Those Doing It

uxaustralia PRO

More Decks by uxaustralia

Other Decks in Design

Featured

Transcript

uxaustralia
PRO