Evolution of Memory in Humans & AI Agents

Presented at AI Lowlands

Transcript

1. Raphael De Lio - Software Engineer (Developer Advocacy) @ Redis The Evolution of Memory in AI Agents And the anatomy of memory in humans 1
2. Raphael De Lio • 8 years implementing distributed systems with Java & Kotlin • Former Dutch Kotlin User Group Leader • Public Speaker Who Am I? 2
3. This is part of my journey diving deep into arti fi cial intelligence throughout the past year. 3
4. Agenda • The anatomy of human memory • Memory in Large Language Models • Similarities and di ff erences • How we can enhance AI Agents memory 4
5. The Anatomy of Human Memory 5
6. “I was just stunned. Bill Estes was working on mathematical models of pavlovian conditioning, and what amazed me was that these were probabilistic, stochastic models. Richard Atkinson (h tt ps://www.youtube.com/watch?v=WxWmkpWOe7M) I had been used to deterministic models, and in the 1940s there were very few probabilistic ones. This swept me away, and I ended up doing my PhD with Estes and spent the next ten years on stimulus- sampling theory.”
7. Sensory Store Short-term Store Long-term Store Lost Information Attention Transfer Retrieval Multi-store memory model (Atkinson & Shi ff rin, 1968)
8. Long-term Memory 8
9. Declarative Non declarative Semantic Episodic Procedural Priming Classical Conditioning Non associative Learning Emotional Responses Skeletal Responses Re fl ex Pathways Medial Temporal Lobe Long-term Memory (Larry Squire, 1992) Cerebellum Amygdala Neocortex Striatum Habituation Sensization Diencephalon 9
10. Declarative Semantic Episodic Semantic memory is your memory for facts, meanings, and general knowledge. • knowing that Paris is the capital of France • knowing the meaning of the word “photosynthesis” • knowing that a bicycle has two wheels • knowing how categories relate (dog → animal) Episodic memory is your memory for personal events and experiences. Episodic memories include: • time (when it happened) • place (where it happened) • feelings and details • the context of your experience 10
11. Declarative Semantic Episodic Semantic memory is your memory for facts, meanings, and general knowledge. • knowing that Paris is the capital of France • knowing the meaning of the word “photosynthesis” • knowing that a bicycle has two wheels • knowing how categories relate (dog → animal) Episodic memory is your memory for personal events and experiences. Episodic memories include: • time (when it happened) • place (where it happened) • feelings and details • the context of your experience 11
12. Declarative Non declarative Semantic Episodic Procedural Priming Classical Conditioning Non associative Learning Emotional Responses Skeletal Responses Re fl ex Pathways Medial Temporal Lobe Long-term Memory (Larry Squire, 1992) Cerebellum Amygdala Neocortex Striatum Habituation Sensization Diencephalon 12
13. Non declarative Procedural Priming Classical Conditioning 13 Non associative Learning
14. Non declarative Procedural Procedural memory allows you to perform skills and actions automatically, without needing to think about how to do them. Among them are included cognitive and motor skills. Motor skills: • Riding a bicycle • Typing on a keyboard • Playing a musical instrument Cognitive skills: • Reading smoothly • Chess patterns • Language Patterns Habits & Routines • Tying shoelaces • Taking a shower • Navigating a speci fi c route 14
15. Non declarative Procedural Procedural memory allows you to perform skills and actions automatically, without needing to think about how to do them. Among them are included cognitive and motor skills. Motor skills: • Riding a bicycle • Typing on a keyboard • Playing a musical instrument Cognitive skills: • Reading smoothly • Chess patterns • Language Patterns Habits & Routines • Tying shoelaces • Taking a shower • Navigating a speci fi c route 15
16. Non declarative Procedural Priming Classical Conditioning Non associative Learning 16
17. Non declarative Priming Priming is a type of memory in which a previous experience makes you process the same or related information more quickly and easily, even if you do not consciously remember the earlier experience. • Word priming • Picture priming • Semantic priming • Repetition priming • Everyday object example 17 Marketeers take full advantage of priming: • Showing the brand repeatedly • Pairing the brand with a feeling • Associating the brand with contexts • Associating the brand with words • Associating the brand with smell, audio, images...
18. Non declarative Procedural Priming Classical Conditioning Non associative Learning 18
19. Classical (Pavlovian) Conditioning Emotional Skeletal Emotional responses are automatic feelings that happen because a neutral stimulus became linked with an emotional event. • A perosn hears a dog bark and then gets bitten. Later any dog bark produces a fear reaction • A person eats a snack while watching a speci fi c TV. Next time they watch the same show, they get hungry • A person that gets sick after eating a certain food may get nauseated by the smell of the same food later Skeletal responses are automatic body movements or re fl ex-like actions that happen because a neutral stimulus became linked with a physical event. 19 • Startle response to phone vibration • Blinking when someone raises their hand quickly • A person that gets sick after eating a certain food may get nauseated by the smell of the same food later
20. Non declarative Procedural Priming Classical Conditioning Non associative Learning 20
21. Non associative Learning Habituation Sensitization Habituation is a form of non-associative memory in which the brain learns to respond less to a repeated, harmless stimulus. • You stop noticing the sound of a fan in your room. • You get used to cold pool water • My dog stops responding to its name Sensitization is a form of non-associative memory in which the brain learns to respond more strongly to a repeated or intense stimulus. 21 • After watching a horror movie, every small sound in your house scares you
22. Declarative Non declarative Semantic Episodic Procedural Priming Classical Conditioning Non associative Learning Emotional Responses Skeletal Responses Re fl ex Pathways Medial Temporal Lobe Long-term Memory (Larry Squire, 1992) Cerebellum Amygdala Neocortex Striatum Habituation Sensitization Diencephalon 22
23. Working Memory (Short-term) 23
24. Central Executive Working Memory (Short-term [Baddeley & Hitch, 1974]) Episodic Buffer Phonological Loop Visuospatial Sketchpad Prefrontal Cortex Broca's area Paerietal Lobe Articulatory Loop Wernicke's area Acoustic Store Visuospatial Sketchpad Occipital Lobe Inner Scribe Parietal Lobe Visual Cache
25. Central Executive The central executive is a fl exible system responsible for the control and regulation of cognitive processes. It directs focus and targets information, making working memory and long-term memory work together. It can be thought of as a supervisory system that controls cognitive processes, making sure the short-term store is actively working, and intervenes when they go astray and prevents distractions It has the following functions: • Updating and coding incoming information and replacing old information • Binding information from a number of sources into coherent episodes • Coordination of the slave systems • Shifting between tasks or retrieval strategies • Inhibition, suppressing dominant or automatic responses • Selective attention 25
26. Central Executive Episodic Buffer Phonological Loop Visuospatial Sketchpad Prefrontal Cortex Broca's area Paerietal Lobe Articulatory Loop Wernicke's area Acoustic Store Visuospatial Sketchpad Occipital Lobe Inner Scribe Parietal Lobe Visual Cache Working Memory (Short-term [Baddeley & Hitch, 1974])
27. Phonological Loop The phonological loop as a whole deals with sound or phonological information. It consists of two parts: a short-term phonological store with auditory memory traces that are subject to rapid decay and an articulatory loop component that can revive the memory traces It has the following functions: • Mentally repeating a phone number • Silently reading a sentence by hearing the words in our "inner voice" • Following speech • Remembering spoken instructions 27 • Acoustic Store -> keeps sounds brie fl y • Articulatory Loop -> your "inner voice" repeating them Acoustic Store Speech Inputs Nonspeech
 Inputs Articulatory Loop
28. Central Executive Episodic Buffer Phonological Loop Visuospatial Sketchpad Prefrontal Cortex Broca's area Paerietal Lobe Articulatory Loop Wernicke's area Acoustic Store Occipital Lobe Inner Scribe Parietal Lobe Visual Cache Working Memory (Short-term [Baddeley & Hitch, 1974])
29. Visuospatial Sketchpad The visuospatial sketchpad deals with visual and spatial information. It consists of two parts: a visual cache that temporarily stores visual details such as shape, color, and form, and an inner scribe component that processes spatial layout, movement, and the arrangement of objects in space. Together, they allow the brain to hold and manipulate mental images for short periods of time. It has the following functions: • Mentally visualizing an object • Remembering where things are • Navigating a space in your mind • Reading maps • Mental Rotation • Tracking Movement • Imagining Patterns and Designs 29 • Visual Cache-> Stores visual detalis • Inner Scribe-> Handles spatial layout & movement
30. Central Executive Episodic Buffer Phonological Loop Visuospatial Sketchpad Prefrontal Cortex Broca's area Paerietal Lobe Articulatory Loop Wernicke's area Acoustic Store Occipital Lobe Inner Scribe Parietal Lobe Visual Cache Working Memory (Short-term [Baddeley & Hitch, 1974])
31. Episodic Buffer The episodic bu ff er is the part of working memory that puts di ff erent types of information together—such as words, images, and past experiences, so you can form one clear idea in your mind. It has the following functions: • Understanding a sentence while reading • Thinking about your dog playing with the ball • Understanding a story someone is telling you • Remembering where you parked the car • Comparing two ideas at the same time 31
32. Central Executive Working Memory (Short-term [Baddeley & Hitch, 1974]) Episodic Buffer Phonological Loop Visuospatial Sketchpad Prefrontal Cortex Broca's area Paerietal Lobe Articulatory Loop Wernicke's area Acoustic Store Visuospatial Sketchpad Occipital Lobe Inner Scribe Parietal Lobe Visual Cache
34. Memory in Large Language Models 34
35. Long term memory in Large Language Models 35 • LLMs learn statistical patterns from large text datasets. • These patterns are encoded in billions of parameters. • Parameters store stable knowledge learned during training. • They do not store exact documents or explicit memories. • The model only updates this long term knowledge through retraining or fi ne tuning.
36. Short term memory in Large Language Models 36 • LLMs do not remember past interactions once the session ends. • They do not retrieve stored memories. • During a single query, they use the input context window as temporary working space. • The attention mechanism lets the model focus on relevant parts of the current input. • This short term memory is erased after the response is generated.
37. Similarities and di ff erences 37
38. Similarities 38 • Both depend on pattern recognition to process information. • Both use context to guide interpretation and recall. • Both show position based e ff ects similar to primacy and recency.
39. Di ff erences 39 • Human memory is dynamic and updates continuously; LLM knowledge is fi xed after training. • Humans store memories selectively; LLMs store statistical patterns without relevance or emotion. • Humans forget adaptively; LLMs do not forget unless retrained. • Human memory is tied to consciousness and intent; LLMs have neither. • Human recall is in fl uenced by emotion; LLM outputs follow statistical likelihood only. • LLM “forgetting” in long conversations is a technical limit, not a cognitive process.
40. The man who could not remember 40
41. Enhancing Agent Memory 41
42. “If neuroscientists gave us be tt er input, it would o ff er real constraints and guardrails for future models. I’m hopeful that arti fi cial intelligence will uncover arithmetics or algorithms humans use in problem-solving that we have overlooked. John McCarthy, the founder of AI, was a close colleague at Stanford, and our research groups shared the PDP-1, the fi rst transistorized computer on the West Coast.” Richard Atkinson (h tt ps://www.youtube.com/watch?v=WxWmkpWOe7M)
43. "It's not because an airplane doesn't fl y like a bird that it means it doesn't fl y." 43
44. Agent Memory Server Agent Memory Server Model (Brookins, 2025) Working Memory Long-term Memory Semantic Episodic
45. Challenges 45 Memory Acquisition Memory Management Memory Retrieval
46. Redis Agent Memory Server 46
47. 47 - Works with any AI model: REST API and MCP interfaces compatible with OpenAI, Anthropic, and others - Remembers everything: Stores conversation history, user preferences, and important facts across sessions - Finds relevant context: Uses semantic search to surface the right information at the right time - Gets smarter over time: Automatically extracts, organizes, and deduplicates memories from interactions Agent Memory Server Redis Agent Memory Server is an open source production-ready memory system for AI agents and applications that:
48. 48 Memory Integration Pa tt erns The most common question developers have is: "How do I actually get memories into and out of my LLM?" Redis Agent Memory Server provides three distinct patterns for integrating memory with your AI applications, each optimized for di ff erent use cases and levels of control.
49. 49 - Session-scoped, durable scratch pad for active conversation state - Stores messages, session data, and structured memories Working Memory Key Features - Durable by default, optional TTL - Automatic window management and summarization - Mixed content: messages, structured memories, context summaries - No indexing; simple JSON stored in Redis What It Stores - Conversation messages - Temporary session data - Structured memories for promotion to long-term storage How Promotion Works - Server extracts summaries or discrete facts - Unpersisted memories are converted into long-term memories with embeddings and stored in the long- term storage
50. 50 - Discrete Strategy: Extract individual facts and preferences (default) - Summary Strategy: Create conversation summaries - Preferences Strategy: Focus on user preferences and characteristics - Custom Strategy: Use domain-speci fi c extraction prompts Memory Extraction Strategies The Redis Agent Memory Server supports con fi gurable memory extraction strategies that determine how memories are extracted from conversations when they are promoted from working memory to long-term storage.
51. 51 Memory Extraction Strategies Discrete Strategy: Extract individual facts and preferences (default)
52. 52 Memory Extraction Strategies Summary Strategy: Create conversation summaries
53. 53 Memory Extraction Strategies Preferences Strategy: Focus on user preferences and characteristics
54. 54 Memory Extraction Strategies Custom Strategy: Use domain-speci fi c extraction prompts
55. 55 - Persistent, cross-session storage for knowledge the agent should keep long term - Functions as the agent’s knowledge base for facts, preferences, and experiences - Survives server restarts and session expiration - Optimized for semantic search, deduplication, and rich metadata Long-term Memory Memory Types 1. Semantic Memory (Facts, preferences, stable knowledge)
 
 Example: “User prefers dark mode interfaces” 2. 2. Episodic Memory (Events with time context)
 
 Example: “User visited Paris in March 2024” Deduplication and Compaction 1. Hash-Based Deduplication
 
 - Removes exact duplicates
 - Keeps the most recent with full metadata 2. Semantic Deduplication
 - Vector similarity identi fi es near-duplicate memories
 - LLM merges related entries when needed 3. Automatic Compaction
 - Cleans up duplicates and merges related records automatically
56. - Manages the full lifecycle of AI memories: creation, promotion, access, aging, forgetting, and compaction - Prevents unbounded growth and maintains performance - All forgetting and compaction handled by background server processes - Contextual grounding: Resolve pronouns and references ("he" → "John") Memory Management Lifecycle Stages - Creation – Memories are created in working memory or directly as long-term items - Promotion – Working memory memories are automatically promoted - Access – System tracks recency and access frequency - Aging – Memories accumulate age and inactivity scores - Forgetting – Background jobs remove old, inactive, or low-priority memories - Compaction – Server merges duplicates and optimizes indexes
57. Memory Management Memory Forgetting - Background tasks evaluate memories periodically - Deletes based on con fi gured thresholds - Uses Redis Docket as a task scheduler Forgetting Policies - Age-based deletion - Inactivity-based deletion - Combined age + inactivity - Budget-based cleanup (keep only top N recent memories)
58. 58 - Semantic similarity: Find memories by meaning, not just keywords - Advanced fi lters: Search by user, session, time, topics, entities - Recency boost: Time-aware ranking that surfaces relevant recent information Intelligent Search
59. 59 - Multiple interfaces: REST API, MCP server, Python client - Authentication: OAuth2/JWT, token-based, or disabled for development - Scalable storage: Redis (default), Pinecone, Chroma, PostgreSQL, and more - Background processing: Async tasks for heavy operations - Multi-tenancy: User and namespace isolation Production Ready
60. 60 Di ff erent Agents may require di ff erent types of memory
61. Software Engineer (Developer Advocacy) Raphael De Lio * Slides GitHub YouTube