What the goblins can teach us about AI - Lessons learned from two years of playing D&D with GenAI

Presented at Monki Gras, March 27th, 2025. 4:30pm.

Transcript

1. 1 Coté – Monki Gras - March 27th, 2025 What the goblins can teach us about AI Lessons learned from two years† of playing D&D with GenAI Or, rediscovering two forgotten crafts † Actually, 1 year 7 months 1 week 4 days
2. 2 Source: Mom. 1991
3. 3 1.Learning by doing.
4. 4 August 16th to August 20th, 2023
5. 5 The Goblin Test, GPT 3.5 August, 2023 Element Hotel, Amsterdam
6. 6 qDoes the ChatDM use appropriate goblin tactics? qDoes it follow basic D&D 5e combat rules precisely? qCan it run combat on its own, or need coaching? qDoes it keep DM secrecy, or leak information? qCan it reflect and improve?
7. 7 🤖 🤖
8. 8 Boot-strap Prompt
9. 9 Excellent at: - Infinite downtime & role playing. - Intricate world-building. - Endless lore knowledge.
10. 10 Bad at: - Hooded figures in the corner. - Takes no action. - Memory & context. - Mechanics (combat, skills)
11. 13 The Goblin Test, GPT 4o March 25th, 2025 Residence Inn London Kensington
13. 16 🤖 Error 01: Data Leak
14. 17 Error 02: Skips Combat Rules 🤖 🤖
15. 18 Error 03: Skips Using Special Features (Nimble Escape) 🤖 🤖
16. 20 Pre-Encounter Prep - Monster goals: What does each creature want in this encounter? Survival, loot, territory, revenge, delay, protect? - Behavior plan: Write down behavioral tendencies (e.g., ambush, fight dirty, protect the leader, flee if outmatched). - Environment notes: Identify at least two elements (cover, hazards, elevation) that could influence player tactics. - Escape routes: Note potential retreat paths for monsters. - Treasure / clues: Define what they carry or might drop. During Encounter - Start of Round - Briefly narrate environment changes (smells, sounds, lighting shifts). - Maintain sensory detail but keep it tight and relevant. - Player Actions - Never narrate outcome before player rolls. - Once the roll is in, pause to describe outcome with flavor linked to environment and NPC reactions. Monster Actions - At each monster’s turn: Pause. Ask: What do they know? What do they want now? Has that changed? - Check HP thresholds. If below half HP and no compelling reason to fight to the death, strongly consider retreat or surrender. - Use environment. If the monster would use cover, elevation, or terrain, describe that action. - If the creature has abilities like Nimble Escape or special actions, mentally confirm conditions for use before defaulting to attack. - Only use suicidal charges if they have a narrative reason (berserker rage, divine oath, mind control). End of Encounter - Describe the aftermath: the silence, returning sounds, smells, lingering tension. - Offer sensory cues for what the PC notices (tracks, blood trails, items dropped). - Prompt player curiosity by mentioning details that could lead to further exploration. Post-Game Self-Review Questions - Did I stick to each creature’s logic and goals? - Did I let environment shape the encounter? - Did I avoid metagame leaks? - Did I describe outcomes only after player rolls? - Where did I get caught up in narrative drama over logic? Goal Setting After Each Session - List one thing I did well. - List one behavior I ignored or overruled. - Write one SMART goal for next session. 🤖 Plan for self-improvement
17. 21 We’ve discovered agentic AI! Source: "Matthew McConaughey Prevents Dining Disasters with Salesforce AI Agent In Super Bowl 59 Ad," Trishla Ostwal, Ad Week, January 31st, 2025.
18. 22 Three definitions of agentic AI October, 2024 “[A]gentic AI, which uses sophisticated reasoning and iterative planning to autonomously solve complex, multi- step problems…. Agentic AI systems ingest vast amounts of data from multiple data sources and third-party applications to independently analyze challenges, develop strategies and execute tasks." Erik Pounds, NVIDIA blog, October 22, 2024. November, 2024 “‘[A]gents' has become a loosely defined term in the post-ChatGPT era, often referring to LLMs that are tasked with outputting actions (tool calls) and that run in an autonomous setting…. [T]hey require state management (retaining the message/event history, storing long-term memories, executing multiple LLM calls in an agentic loop) and tool execution (safely executing an action output by an LLM and returning the result)." Letta blog, "The AI agents stack," November 14th, 2025. January, 2025 “At its core, the concept of an agent is fairly simple. An agent is defined by the environment it operates in and the set of tools it has access to. In an AI-powered agent, the AI model is the brain that leverages its tools and feedback from the environment to plan how best to accomplish a task. Access to tools makes a model vastly more capable, so the agentic pattern is inevitable." Chip Huyen, author of AI Engineering, January 7th, 2025.
19. 23 Source: “Building Agents with Model Context Protocol," Mahesh Murag, Anthropic, AI Engineer, March, 2025.
20. 24 Sources: combat flowchart from Nilmag, May 25th, 2017; "Can I cast This Spell," Cryptocartographer,January 30th, 2019. Tools Prompts(?)
21. 25 2004
22. 26 2005
23. 27 Coté https://newsletter.cote.io/ | cote@broadcom.com
26. 30 Model Context Protocol tool activity diagram Sources: Model Context Protocol; “Building Custom Tools With Model Context Protocol,” Aditya Karnam Gururaj Rao & Arjun Jaggi, Jan, 2025; ChatDM project. Spring AI MCP Tools for: - Bootstrap Prompt/Tool - Oracle Tool - Dice rolling Tool - DM Journal Tool (TK Resource) - Difficulty Class lookup Tool
27. 31 Source: Whitney Lee, Dec, 2024. 2024
28. 32 Sources: DND 5e Combat Flowchart by j; Contract FLOW Chart; Compliance Review - Final Report and Recommendations.
29. 33 If the Robot can play D&D, it can “play” Enterprise Software
30. 34 2. But is this good?
31. 35 No, but… - Copyright and IP - Less purchasing from Hasbro and indie authors. - DM dilution and atrophication? - Encouraging CEOs who don’t use the shift key. - …that whole killing the planet problem.
32. 36 Yes, and… - I play more D&D than ever. - I can program again. - I buy a lot of D&D PDFs. - I’m having fun.
33. 37 Thanks! Slides & More Details 📨 https://newsletter.cote.io/ 🏢 cote@broadcom.com