Chasing Engaging Ingredients in Design

Talk at the BIG Bristol Interaction Group on April 3, 2025.

Transcript

1. Chasing Engaging Ingredients in Design Prof Sebastian Deterding Dyson School of Design Engineering Imperial College London @dingstweets @codingconduct@hci.social
2. Work Learning Health Civics Leisure Environment A universal problem: affording motivation
3. New societal challenges … … require motivating collective behaviour change
4. How do we help designers create systems that reliably motivate?
5. <1> A fable
6. I want to know how to bake a delicious cake. !
7. WELCOME TO AEM FHI 2023 The Leading Food-Human-Interaction Conference FHI23
8. Coating required? The Impact of Patisserieness on Taste in Baked Goods
9. low patisserieness medium patisserieness high patisserieness N=128 eaters, inherent tastiness inventory (ITI) Patisserieness positively affects tastiness, p<0.05
10. 12.2 Implications for Cooking 12.2 Implications for Cooking … “For more tasty baked goods, consider a higher degree of patisserieness, such as cupcakes or cakes.” …
11. "
12. Hi, I’m a pastry chef and read your paper. Can I ask you a question? Sure, my pleasure. How do I bake a delicious cake? Our results show that people find high patisserieness very tasty. So how do I do that? Have you thought about baking a cake? Yes. I’m already making a cake. How do I make a delicious cake? Well, make it … very caky I guess?
13. #
15. Different epistemic cultures
16. Science episteme Analyse and abstract real multitudes into decontextualised, universal building blocks, described in data and diagrams.
17. Science episteme Design techne/phronesis Analyse and abstract real multitudes into decontextualised, universal building blocks, described in data and diagrams. Synthesise and specify a single functioning whole within the local context, crafted from objects and actions. ?
18. We literally see the world differently.
19. What psychologists see Abstract analytic universal models in the brain
20. What designers see Concrete specific artefacts in the world
21. One solution, many names: Active ingredients in design! Leaderboard sparks competition Points reward activity Progress bar creates sense of progression Michie et al., 2013
22. Löwgren, 2013; Dalsgaard & Dindler, 2014; Colusso et al., 2017, 2019 Active ingredients are “intermediate knowledge forms” bridging theory and artefact
23. Game design elements Nudges Behaviour change techniques Psychostructural characteristics Motivational affordances Rhetorical tropes Collative variables Persuasive patterns Dark patterns Patterns This has been an intuition in many domains beyond games
24. Naive me, ca. 2013: Count me in! How hard can this be?
25. It’s hard. We need formal models of motivational affordances – and games are a great way to get them Wise old me, ca. 2025
26. What I mean with formal models* Turning verbal theories into executable formulas * Yes, modelling cognition is itself an intractable problem (van Rooj et al., 2021), but “all models are wrong; some are useful”. Or: “Models are stupid, and we need more of them” (Smaldino, 2017)
27. Theorising is a generating and constraining cycle between observing, formalising, analysing, and testing Theory Formal Verbal Empirical testing Analytic testing Naturalistic observation Heavily inspired by Conceptual analysis Formal analysis Simulation Severe, quantitative hypothesis-testing Phenomenology Ethnography Grounded theory
28. But how does that help designers create systems that reliably motivate?
29. <2> Problems
30. #1 Psychology: How does motivation work? #2 Design: What affords motivation? How do we create systems that reliably motivate? 2 problems, 2 opportunities for formal theory
31. #1 Psychology: How does motivation work? The Replication Crisis hits most research findings … 2005 theory: Most findings are false 2015+ data: 39% of studies replicate, effect sizes half
32. Oberauer & Lewandowsky, 2019; Eronen & Bringman, 2021; Fried, 2021; Muthukrishna & Henrich, 2019 … pointing toward a deeper theory crisis • New open science practices don’t suffice for cumulative progress because … • Current theories are mostly ‘weak’ verbal theories that don’t allow unambiguous derivation of hypotheses or severe testing • Verbal constructs cannot resolve jingle-jangle • Most verbal theories label effects instead of explaining them
33. But surely, not in motivation and HCI? $
34. Take Balanced Challenge and Competence/Mastery/Flow An effect widely recognised in practice and research “Challenge is consistently identified as the most important aspect of good game design. Games should be sufficiently challenging, match the player’s skill level” Sweetser & Wyeth, 2005, 6
35. Challenge as difficulty-skill balance Enjoyment Balanced challenge is a core affordance in self-determination theory, yet poorly specified »Within SDT, then, optimal challenge means facing demands that most often one can master, rather than ones that are continuously at the leading edge of one’s capabilities. That type of high difficulty challenge should, however, be an intermittent element, in which case it can enhance and heighten intrinsic motivation« Deci & Ryan, 2017, 153
36. Challenge as difficulty-skill balance Enjoyment • How do you quantify “difficulty” and “skill”? • Linear or log scales or …? • What shape is that curve? • What equation describes the shape of this curve? • Evidence is mixed, operationalisations are “too heterogeneous to aggregate” Fong et al, 2015, Cutting et al, 2023 Balanced challenge is a core affordance in self-determination theory, yet poorly specified
37. Cutting et al., 2022 We found this when we tested the effect of “balanced challenge” on enjoyment and engagement
38. No significant difference in enjoyment or engagement, no ‘goldilocks’ optimum around balanced challenge (n=311) Cutting et al., 2023
39. Players perceive draw as most balanced, enjoy easy, and seek hard – which fits and contradicts SDT?!? Deterding & Cutting, 2023 Deterding & Cutting, 2023
40. • Effectance: Causing effects (even unintended, must be novel) • Task performance: Attaining some defined goal • Capacity growth: Increasing quality or range of skills Deterding et al, 2025 Competence as defined in SDT has at least 3 meanings that need not empirically co-occur »Competence refers to feeling effective in one’s interactions with the social environment—that is, experiencing opportunities and support for the exercise, expansion, and expression of one’s capacities and talents. Where individuals are prevented from developing skills, understanding, or mastery, the competence need will be unmet« Deci & Ryan, 2017, 86
41. Deterding et al, 2025 Present self-report scales and experiments mostly reflect only task performance Intrinsic Motivation Inventory (IMI) 1. I think I am pretty good at this activity. 2. I think I did pretty well at this activity, compared to other students. 3. After working at this activity for a while, I felt pretty competent. 4. I am satisfied with my performance at this task. 5. I was pretty skilled at this activity. 6. This was an activity that I couldn’t do very well. (R) Basic Psychological Need Satisfaction and Frustration Scale (BPNSFS) 1. I feel confident that I can do things well. 2. I feel capable at what I do. 3. I feel competent to achieve my goals. 4. I feel I can successfully complete difficult tasks. Basic Psychological Need Satisfaction Scale (BPNSS) 1. Often, I do not feel very competent. (R) 2. People I know tell me I am good at what I do. 3. I have been able to learn interesting new skills recently. 4. Most days I feel a sense of accomplishment from what I do. 5. In my life I do not get much of a chance to show how capable I am. (R) 6. I often do not feel very capable. (R)
42. Deterding et al, 2025 SDT claims success feedback increases competence and competence generates challenge-seeking: a contradiction
43. Verbal theory + standard factor analytic methods can’t resolve illusory essences, lexical fallacies, jingle-jangle • Immersion ≈ engrossment ≈ flow ≈ absorption ≈ involvement ≈…? • Curiosity ≈ knowledge-seeking ≈ novelty-seeking ≈ interest ≈ …? • Competence ≈ mastery ≈ effectance ≈ agency ≈ flow ≈ …?
44. Verbal theories often merely state and label effects instead of providing mechanisms that explain them • “Engines cause movement” • “Dormative properties cause sleep” • “Grit causes behavioural persistence” • “Competence causes challenge- seeking” • … don’t explain which constituent mechanisms work how to bring this effect about
45. If you want to know more, read these preprints osf.io/preprints/psyarxiv/n6x8s_v2 arxiv.org/abs/2502.07423
46. #2 Design: What affords motivation? Leaderboard sparks competition Points reward activity Progress bar creates sense of progression Michie et al., 2013
47. Surely, we have made more progress here? %
48. Kristal & Whilans, 2020; Bird et al., 2021; Löschel et al., 2020 No, again, the replication crisis rears its head
49. (Aside from fraud …)
50. Also for game design elements: They can work … Höchsmann et al., 2019
51. Longitudinal (6m) RCT, n=69, with major app Fitocracy Intrinsic motivation does increase activity But: No significant differences in physical activity, body fat %, intrinsic motivation between gamification and control group Main issue: Non-use/quick disengagement Gummelt, 2017 … and they can not work (or even backfire)
52. Almost every new gamification paper I read
53. “A muddle of things we here call ‘design elements X,Y,Z’ affect a muddle of things we here call ‘engagement’, unless when they don’t. More research is needed.” The default contribution
54. Goveia, Pereira, Karapanos et al., 2016 #1 Design detail matters: Patterns remain underspecified
55. Codecademy van Roy, Deterding & Zaman, 2019 Khan Academy #2 Different understandings, different uses and effects
56. Deterding, 2016 Standard wisdom: (Narrative) choice supports autonomy Game journalist, faced with seeing game endings under deadline “Sometimes, you just have to play, you have to get further, and then, … playing is indeed work” #3 Context shapes effects
57. In short: Motivation is afforded by whole functional person-environment mechanisms Lewin, 1936; Gibson, 1977; Warren, 1984; Deterding, 2011 B=f(P,E)
58. Yet current practice mostly identifies surface patterns, imputing but not specifying or validating causal mechanisms Deterding, 2015
59. Slack onboarding bot ≈ Early bot levels in WoW Game-inspired bot caused Slack’s success Yet current practice mostly identifies surface patterns, imputing but not specifying or validating causal mechanisms Deterding, 2015
60. This results in cargo cult design that copies non-functional or brittle, underspecified surface patterns Deterding, 2011, 2014
61. In short, active ingredients are underspecified, like motivational theories
62. To get a better analytic and practical grip, we need affordances as dynamic person-environment relations … Deterding, 2011 Kidd, Piantadosi & Allan, 2012 Warren, 1984
63. … at the grain of stable necessary and sufficient actor- environment elements that make a functioning mechanism
64. <3> Solutions
65. So how do we specify motivational affordances?
66. You probably guessed it Theory Formal Verbal Empirical testing Analytic testing Naturalistic observation
67. Engaging creative practitioners and creative practice tunes us into professional vision, concepts, material ‘grain’ Wang & Deterding, under review; Wang & Deterding, in preparation Löwgren, 2013
68. • Task performance • Doing better/worse than expected Observation surfaces constraints, likely elements, mechanisms Ballou & Deterding, 2023; Petralito et al., 2017 Theory Formal Verbal Empirical testing Analytic testing Naturalistic observation
69. Conceptual analysis foregrounds inconsistencies, possible fits Deterding et al., 2022 Theory Formal Verbal Empirical testing Analytic testing Naturalistic observation • If competence = doing well, people will choose games with certain success • Better than expected progress doesn’t fit SDT, but PP
70. Formalisation forces us to specify necessary & sufficient components to render them severely testable Kidd, Piantadosi & Allan, 2012 Warren, 1984 Theory Formal Verbal Empirical testing Analytic testing Naturalistic observation
71. Computational intrinsic motivation offers ready formalisms - for simulation and working RL agents Oudeyer & Kaplan, 2007; Lintunen, Ady, Deterding, & Guckelsberger, 2025
72. They run in games and virtual environments and include and exceed formalisms in, e.g., active inference/PP Biehl et al., 2018; Guckelsberger et al., 2017
73. Games and virtual environments can be excellent petri dishes for intrinsically motivated behaviours Theory Formal Verbal Empirical testing Analytic testing Naturalistic observation
74. #1 Games bring rich longitudinal telemetry of behaviour and environment telemetry for model exploration & testing Zendle et al., 2023 – even better when fused with psychometrics
75. All game trace data is a GDPR request away – and the Smart Data Donation Service will make it easy sdds.ac.uk
76. #2 Games allow large-scale human/model agent comparison in dynamic, rich, ecologically valid environments King, 2021; Microsoft Research, 2017
77. #2 Games allow large-scale human/model agent comparison in dynamic, rich, ecologically valid environments King, 2021; Microsoft Research, 2017
78. E.g., testing surprise formalisms to explain novelty/variety effects on engagement within and across game levels Modirshanechi, Brea, & Gerstner (2022): A taxonomy of surprise definitions
79. #3 Large online (multi-cultural) metastudies allow determining limits of generalisability – and more Baribault et al., 2018
80. Generative design lets us systematically create and dynamically adapt hypervariate design spaces Agent behaviour Procedural content generation Industrial design Narrative Architecture Visual art Music
81. Hypervariate online studies using generated large n design spaces can test actual parameter variants and interactions
82. #4 Every game AI requires/embodies some implicit or explicit model of what motivates players Procedural content generation Difficulty adjustment/matchmaking Automated playtesting AI NPCs AI-assisted design tools AI directors
83. <4> Summary
84. Reliably designing for motivation is an important problem. !
85. Kristal & Whilans, 2020; Bird et al., 2021; Löschel et al., 2020 But current research and practice remain hit and miss
86. … because theories and design elements are underspecified Challenge as difficulty-skill balance Enjoyment
87. Better theorising can help overcome this crisis … Theory Formal Verbal Empirical testing Analytic testing Naturalistic observation
88. … employing formal modelling informed by computational intrinsic motivation … Oudeyer & Kaplan, 2007; Lintunen, Ady, Deterding, & Guckelsberger, 2025
89. … and making use of today’s generative game design tools and online environments Procedural content generation Difficulty adjustment/matchmaking Automated playtesting AI NPCs AI-assisted design tools AI directors
90. Thank You! Prof Sebastian Deterding s.deterding@imperial.ac.uk codingconduct.cc @ bsky osf.io/preprints/psyarxiv/n6x8s_v2 arxiv.org/abs/2502.07423