Product Strategy Agility: How to Use Experiments and Options to Create Products Your Customers Love

Senior leaders often want to see months- or years-long product roadmaps. But these predictions often do not create products your customers will love. While customers aren’t fickle, they often do not know what they want until you give them something to try. That means product leaders need to integrate experiments and options into their roadmaps.

In this presentation, Johanna Rothman will explain:

- How to limit the duration of a roadmap and show possible options.
- Clarify the three ideas of experiments including defining what to measure, how long to experiment, and when to learn from the experiment.
- What to do when your customers react differently to your experiment, including when half your customers love the feature and the other half hate it.
- How to set expectations for senior leaders and customers for when the roadmap will change.

Transcript

5. Product Strategy Agility: How to Use Experiments and Options to Create Products Your Customers Love Johanna Rothman jr@jrothman.com www.jrothman.com https://mastodon.sdf.org/@johannarothman
6. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman A Story from a Startup Searching for the “Right” Product Fit 6
7. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman The Time-Based “Everything Forever” Roadmap 7
8. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman More “Everything Forever” Roadmaps 8
9. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Why We Have “Everything” Roadmaps • No one loves ambiguity • But… • Customers don’t know what they want until they see something • Overworked product managers • Product strategy agility allows us to respond to new information (& create new roadmaps) 9
10. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Our managers don’t trust us to deliver anything, so they ask for everything 10
11. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Don’t blame anyone Instead, look at the feedback loop 11
12. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Premature Commitment Reinforcing Feedback Loop • Push planning creates premature overcommitment • That overcommitment creates a false precision about the future • We think we must do “everything” leading to bloated products • “Just continue” without re-examining the product strategy 12
13. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Too Many Disappointed People • Customers, when the public roadmap deviates • Teams because they feel pressure to be feature factories & not assess the value of the work to the customers • Senior management when teams “miss” a “deadline” 13
14. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Maybe people don’t ❤ roadmaps? What can we do instead? 14
15. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman A more agile product planning approach: Experiments and options: oriented to a product goal 15
16. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Our Agenda 1. A little about product goals 2. Limit the roadmap duration with experiments and options 3. How to conduct an experiment and when customers react in unexpected ways 4. Set expectations about change for senior leaders and customers 5. Where you might start 16
17. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman 1. Product Goals Offer Value to the Customer • Too many product goals are about the bene fi t to the organization • But product goals have to be about the customer bene fi t/value • When teams deliver value to the customer, the organization wins 17
18. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Candidate Product Goals • Who does this product serve? • Ideal customers • Secondary customers • What do those people need now? • How does that increase product value now and possibly in the future? 18
19. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman 2. Limit the Duration of the Roadmap • Take a risk-based approach (so you can bring management along) • Project risks? Can the team do this? (the project pyramid here) • Product risks? How much innovation? • Portfolio risks? Can we innovate in time to have a useful product? 19
20. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Product Risks: When to Experiment • Product risks clarify how much innovation you need and when • Very few product ideas survive customer contact 20
21. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Effects of Increased Risks • We will need to learn faster (shorter team-based feedback loops) • We will change the roadmap because we will learn • As a result, we must stay nimble and incorporate agility into planning 21
22. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Change the Roadmap & Words • From features to problems, experiments, and options • How little can you describe in a roadmap vs how much product description do you need? • Limit the WIP to “5” per team (or “5” feature sets for a program) • A random but small number. I prefer 3. • Keep all roadmaps to a maximum of a quarter 22
23. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman A Team-Based Initial Roadmap • MVP = Minimum Viable Product • MVE = Minimum Viable Experiment • MMF = Minimum Marketable Feature set • Each of these solves speci fi c customer problems, but not the entire product 23
24. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Longer Roadmap for Managers (If You Must) 24 Commitments Options: Pull from these when the above is complete Everything below the Big Black Line is a possibility, not a requirement
25. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Separate Options from Commitments • Big black line separates the commitments (above) from the possibilities (below) • Fewer things, more answers in each quarter • We might not need the options below the big black line • We won’t know until we get customer feedback from the experiments 25
26. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman 3. Experiments • What do you need to know now? Form a hypothesis • How little can you do to support that need? Experiment • How will you know you’ve learned enough? Gather data • Assess what you learned 26
27. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Measures Do NOT Have to Be Perfect • Most experiments require fast learning • Gather suf fi cient, not perfect measures • Leading indicators where possible • Might have to watch customers use the product 27
28. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman An Example Experiment • In a port (no innovation), team discovered some “critical” features not working • Could they release without those features? • Experiment: • Hypothesis: Release ASAP for their best customers • Measure: Qualitative customer feedback (“Will you be a reference account?”) • Results: Yes, for 4, No for 3 (“not enough done yet”) • Learn: Finish more, but not much more • End: Released four months earlier than the expected year 28
29. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman When Customer Feedback Is Not Suf fi cient • What if the previous example had 100 customers and 50 said, “ship it” and the other 50 said, “no” • Review the product goal and the ideal customers • What customers do we want to attract? What business value do we want to offer those people? • Why these customers, now? 29
30. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Options Allow You to Replan Based on Feedback • “Plans are worthless, but planning is everything” • We have suppositions about the product and the customers and the market • We need feedback from experiments to evaluate those suppositions • We want to change this overcommit loop to a reasonable commit loop 30
31. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman 4. Set Expectations for Change • Shorter roadmaps set frequent change expectations • We know less early in development • Little’s Law and Cycle Time 31
32. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Little’s Law • The higher the WIP, the longer the work takes (cycle time) • Long roadmaps create huge WIP • The shorter the cycle time, assuming a reasonable arrival rate, the faster the throughput • Keep work small. Finish what you start. • Don’t plan more than you can see fi nishing 32 Work in Progress = (WIP) Cycle Time * Throughput
33. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman 5. Start Here • “How little” for all plans and planning • Cycle time is your friend • Use your customers for experiments (carefully!) • Replanning is good & prevents bloat • Learn from development and experiments and incorporate feedback into the plans 33
34. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Avoid Extensive Push-Plans • You don’t need extensive plans • Clarify the corporate strategy so you know why this product now (not part of this webinar) • Clarify the product strategy so you know why these customers now • Always ask the “how little can we do before we plan next?” • Deliver small chunks of value so you can decide what’s next 34
35. © 2024 Johanna Rothman https://mastodon.sdf.org/@johannarothman Let’s Stay in Touch • Pragmatic Manager: • www.jrothman.com/ pragmaticmanager • Please link with me on LinkedIn 35