The Only Good Quality Metric is Morale

Every metric for a QA team has pitfalls.

Some are combative and drive a wedge between teams. Some are useless or easily gamed. Some don’t make any sense at all. So what’s an agile team to do if they want to measure the quality of the software they are working so hard to produce?

Jenny Bramble suggests looking inward, past all the obvious metrics and into the heart of the team. The team’s morale can accurately predict the quality of the software they produce. In this talk, she will discuss some common metrics and their pit falls before making the case for morale as the top QA metric as well as how to measure changes in morale. She’ll show how to measure changes in morale over time and what you can do to help increase the morale–and thus quality!–of your team.

Transcript

1. THE ONLY GOOD QUALITY METRIC IS MORALE @jennydoesthings
2. @JENNYDOESTHINGS HI I’M JENNY! ▸ Test Lead at Willowtree ▸ Yes, we’re hiring! ▸ Human interfacing is my favorite thing ▸ Testing is also my favorite thing ▸ Two cats—Dante and Dax ▸ Ask me for a sticker later! ▸ Yes, they’re also my favorite things ▸ My pronouns are she/her
3. IT’S STORY TIME! Me, probably @JENNYDOESTHINGS
4. THIS QUESTION HAS PLAGUED ME FOR MY ENTIRE CAREER SERIOUSLY @jennydoesthings
5. @JENNYDOESTHINGS SO, LET’S TALK ABOUT IT ▸Less effective metrics ▸What do you actually want to measure? ▸The case for morale as a metric ▸Measuring morale
6. LESS EFFECTIVE METRICS BABY DON’T HURT ME DON’T HURT ME NO MORE @jennydoesthings
7. WHO DO YOUR METRICS HURT? ASK YOURSELF THIS: @jennydoesthings
8. WHAT IS THE CONTEXT FOR YOUR METRICS? ASK YOURSELF THIS: @jennydoesthings
9. @JENNYDOESTHINGS LESS EFFECTIVE METRICS - WHAT ARE THEY ▸ Test cases executed ▸ Bugs found ▸ Bugs resolved ▸ Deferred defects ▸ X of Y completed
10. ANYTHING THAT IS A STRICT COUNT CAN BE LESS EFFECTIVE THE SHORT STORY IS… @jennydoesthings
11. @JENNYDOESTHINGS LESS EFFECTIVE METRICS - WHY ARE THEY LESS EFFECTIVE ▸ Very game-able ▸ Encourages dishonest reporting ▸ Pit parts of your team against each other ▸ Enhances toxic patterns in teams ▸ Most of them measure the process not the product ▸ They are inaccurate…depending on what you want to measure
12. @JENNYDOESTHINGS LESS EFFECTIVE METRICS - WHAT DRAWS US TO THEM ▸ Easy to count ▸ Easy to measure ▸ Easy to compare ▸ Easy to explain ▸ Graphs!! Everyone loves graphs!!
13. WE ARE CONDITIONED TO ACCEPTS THESE AS VALID MEASURES OF OUR EFFECTIVENESS IN THE END… @jennydoesthings
14. @JENNYDOESTHINGS LESS EFFECTIVE METRICS - CAVEAT ▸ This assumes you’re primarily wanting to measure the product and the people ▸ If you are looking to measure your process and the flow of defects, these metrics may be exactly what you need ▸ So…
15. WHAT DO YOU WANT TO MEASURE? YOU MEAN I HAVE TO CHOOSE?? @jennydoesthings
16. @JENNYDOESTHINGS WHAT DO YOU WANT TO MEASURE ▸ Numbers? ▸ Work done? ▸ Quality of…? ▸ Satisfaction? ▸ Compliance ▸ Performance ▸ Your process? ▸ Just want some pretty graphs
17. @JENNYDOESTHINGS WHAT DO YOU WANT TO MEASURE ▸ Take time to think about what matters to you ▸ What supports that? ▸ How can you quantify it? ▸ Metrics are hard
18. METRICS ARE HARD REMEMBER: @jennydoesthings
19. YOU GET WHAT YOU MEASURE REMEMBER: @jennydoesthings
20. THE CASE FOR MORALE NOT MORELS OR MORALS @jennydoesthings
21. @JENNYDOESTHINGS THE CASE FOR MORALE - WHAT IS IT ▸ Psychological safety ▸ Emotional health ▸ Contentedness ▸ Delight ▸ Pride ▸ Core Values
22. @JENNYDOESTHINGS THE CASE FOR MORALE - CORE VALUES ▸ Core Values are a set of values that govern a team or company ▸ Some companies have these codified ▸ Sometimes they actually follow them ▸ Sometimes they don’t ▸ Each team should have values they adhere to
23. @JENNYDOESTHINGS THE CASE FOR MORALE - WHY ▸ Not as easy to game without straight up lying ▸ One person is not signaled out ▸ Describes the health of the team ▸ Happy, safe teams makes better products ▸ Retention is much higher on high morale teams
24. MORALE SUPPORTS ALL YOUR MAJOR GOALS MOST OF ALL @jennydoesthings
25. @JENNYDOESTHINGS THE CASE FOR MORALE - WHY ▸ Happy, safe teams consistently out perform unhappy, unsafe teams. ▸ Health of the team correlates to performance ▸ Google study shows psychological safety as the top driver of performance ▸ Safe teams will work together to make the team as a whole better
26. LET’S TALK MEASUREMENTS WAIT, DIDN’T WE JUST DO THIS? @jennydoesthings
27. HOW DO YOU MEASURE FEELINGS? ASK YOURSELF THIS: @jennydoesthings
28. HOW DO YOU PUT THEM INTO CONTEXT? ASK YOURSELF THIS: @jennydoesthings
29. @JENNYDOESTHINGS THE CASE FOR MORALE - CONTEXT ▸ Pay attention ▸ One-on-ones ▸ That One Person ▸ Company wide initiatives ▸ Glassdoor
30. CURATE SAFE SPACES TASK: @jennydoesthings
31. FORM A SURVEY TASK: @jennydoesthings
32. @JENNYDOESTHINGS LET’S TALK MEASUREMENTS - SURVEYS ▸ Your team needs to be psychologically safe to participate ▸ You need a team of large enough size to be anonymous ▸ Or agree with the team that it is okay if it’s clear who is answering ▸ Surveys need to be done regularly ▸ Actions must be taken on the results ▸ Do not focus on individuals, focus on the team
33. @JENNYDOESTHINGS LET’S TALK MEASUREMENTS - SURVEYS ▸ Run the surveys regularly - ie: every sprint ▸ Score questions consistently ▸ 1-5, 1-7 ▸ Strongly agree to strongly disagree ▸ Keep an eye on the trends of the team, but don’t place too much stock until you have enough data
34. WHAT DO YOU ASK? OKAY, SO… @jennydoesthings
35. @JENNYDOESTHINGS SURVEYS - THE GOOGLE QUESTIONS ▸ If you make a mistake on this team, it is often held against you. ▸ Members of this team are able to bring up problems and tough issues. ▸ People on this team sometimes reject others for being different ▸ It is safe to take a risk on this team. ▸ It is difficult to ask other members of this team for help. ▸ No one on this ream would deliberately act in a way that undermines my efforts. ▸ Working with members of this team, my unique talents are valued and utilized.
36. @JENNYDOESTHINGS SURVEYS - OTHER QUESTIONS/PHRASINGS ▸ I am comfortable asking for help within my team. ▸ I feel I can fail openly without being shamed by my team. ▸ I can express ideas that might be contrary to the rest of the team. ▸ I like my team. ▸ I am not comfortable giving or asking for constructive feedback with my team
37. @JENNYDOESTHINGS SURVEYS - OTHER QUESTIONS/PHRASINGS ▸ We can brainstorm together or in front of each other. ▸ My team has a shared purpose or goal. ▸ We understand how we work together as a team. ▸ The team collaborates well together. ▸ My team has strong communication. ▸ No one is left out of the team.
38. RELATE TO QUALITY? SO HOW DOES THAT @jennydoesthings
39. ANY QUESTIONS? DERP @jennydoesthings
40. @JENNYDOESTHINGS FIND ME ONLINE! ▸ jenny.bramble@gmail.com ▸ http://twitter.com/jennydoesthings ▸ https://linkedin.com/in/ jennybramble ▸ http://jennydoesthings.com ▸ Willowtree!