Context Engineering - Making Every Token Count

Slides from my talk at O'Reilly AI CodeCon

Video: https://www.youtube.com/watch?v=zMM5zqesL1g

How do you get the best out of AI systems when every token in the context window matters? In this talk I break down context engineering - the art and science of filling an AI’s limited memory with the right mix of instructions, data, and history so it can perform at its best.

You'll learn:

- What tokens and context windows really are, and why they matter 
- Why prompt engineering often fails without strong context management 
- Practical strategies to avoid vague, hallucinated, or poisoned responses 
- Patterns for context management - Write, Select, Compress, Isolate
- How modern AI tools like Cursor and Cline optimize context automatically 
- Actionable tips for AI-assisted coding: from error logs and design docs to database schemas and PR feedback

Whether you’re building with coding agents, debugging with AI, or designing smarter prompts, this talk will help you make every token count.

Transcript

1. CONTEXT ENGINEERING ADDY OSMANI MAKING EVERY TOKEN COUNT
2. Based on illustration by dexhorthy
3. WHAT ARE TOKENS? https://platform.openai.com/tokenizer
4. WHAT’S A CONTEXT WINDOW?
5. CONTEXT WINDOWS ARE LIKE LIMITED RAM Curation of what fi ts into RAM is analogous to “context engineering” ANALOGY
6. PROMPT ENGINEERING Clear instructions for models so they can accomplish a task https://www.youtube.com/watch?v=ysPbXH0LpIE PRELUDE
7. PROMPT ENGINEERING FAILURES STEM FROM CONTEXT MISMANAGEMENT https://www.instagram.com/p/DMc4ujyTuk6/?img_index=6
8. CONTEXT ENGINEERING MEANS PROVIDING AN AI WITH ALL THE INFORMATION AND TOOLS IT NEEDS TO SUCCESSFULLY COMPLETE A TASK – NOT JUST A CLEVERLY WORDED PROMPT. DEFINITION
9. ADD CONTEXT CONTEXT WINDOW USE CONTEXT IN EDITORS & IDES
10. VISUAL CONTEXT IS PRETTY POWERFUL
11. https://www.youtube.com/watch?v=ysPbXH0LpIE
12. https://context.addy.ie
23. GOOD CONTEXT IS HARD. PROBLEM
24. AI CODING PERFORMANCE OFTEN DIPS WHEN CONTEXT WINDOWS EXCEED ~50% FULLNESS PROBLEM
25. TOO LITTLE CONTEXT: VAGUE OR HALLUCINATED RESPONSES. TOO MUCH CONTEXT: DISTRACTED, UNABLE TO FIND RELEVANT INFO, OVER-INDEXING ON PATTERNS. BAD CONTEXT: POISONING, TRUSTING INCORRECT STATEMENTS OVER TRAINING.
26. TOO LITTLE CONTEXT: VAGUE OR HALLUCINATED RESPONSES. TOO MUCH CONTEXT: DISTRACTED, UNABLE TO FIND RELEVANT INFO, OVER-INDEXING ON PATTERNS. BAD CONTEXT: POISONING, TRUSTING INCORRECT STATEMENTS OVER TRAINING.
27. TOO LITTLE CONTEXT: VAGUE OR HALLUCINATED RESPONSES. TOO MUCH CONTEXT: DISTRACTED, UNABLE TO FIND RELEVANT INFO, OVER-INDEXING ON PATTERNS. BAD CONTEXT: POISONING, TRUSTING INCORRECT STATEMENTS OVER TRAINING.
28. PATTERNS FOR CONTEXT MANAGEMENT https://blog.langchain.com/context-engineering-for-agents/
29. WRITE CONTEXT • Long-term memories (across sessions) • Scratchpad (within session) • State (within sessions) @aifolksorg
30. SELECT CONTEXT • Retrieve relevant tools • Retrieve from scratchpad • Retrieve long-term memory • Retrieve relevant knowledge. @aifolksorg
31. COMPRESS CONTEXT • Summarize context to retain relevant tokens • Trim to remove irrelevant tokens @aifolksorg
32. ISOLATE CONTEXT • Partition context in state • Hold in environment/sandbox • Partition across multi-agents @aifolksorg
33. AUTOMATIC CONTEXT OPTIMIZATION CURSOR CLINE
34. HOW TO FIX YOUR CONTEXT https://www.dbreunig.com/2025/06/26/how-to- fi x-your-context.html
36. •Be precise: Vague requests lead to vague answers. The more specific you are, the better your results will be. •Provide relevant code: Share the specific files, folders, or code snippets that are central to your request. •Include design documents: Paste or attach sections from relevant design docs to give the AI the bigger picture. •Share full error logs: For debugging, always provide the complete error message and any relevant logs or stack traces. •Show database schemas: When working with databases, a screenshot of the schema helps the AI generate accurate code for data interaction. •Use PR feedback: Comments from a pull request make for context-rich prompts. •Give examples: Show an example of what you want the final output to look like. •State your constraints: Clearly list any requirements, such as libraries to use, patterns to follow, or things to avoid. CONTEXT ENGINEERING FOR AI CODERS
37. CONTEXT IT’S NOT THE PROMPT, IT’S THE ADDYOSMANI.COM