Weak References in Go 1.24: Memory Management Superpowers

Presented at GoLab 2025 in Florence, Italy on October 7th, 2025.
https://golab.io/talks/weak-references-in-go-1-24-memory-management-superpowers

Transcript

1. 1 October 7th, 2025 GOLAB Naoki Kuroda Weak References in Go 1.24: Memory Management Superpowers
2. 2 • Software Engineer @ CyberAgent, Inc. • Developer Productivity team • Building Bucketeer - Open-source feature flag system Self-Introduction
3. 3 Agenda 1. What are Weak References? 2. The Journey to Weak References 3. Internal Implementation: Core Concepts 4. Practical Patterns and Caveats 5. Summary
4. 4 What are Weak References?
5. 5 What are weak references in the first place?
6. 6 The Essential Difference Strong Reference ◦ A reference that keeps the object alive ◦ As long as there's a strong reference, GC won't collect the object ◦ Regular Go pointers and variable references fall into this category Weak Reference (weak pointer) ◦ When an object is only referenced by weak references, GC considers it eligible for collection ◦ Objects accessible only through weak references can be freed by GC at any time ◦ When an object is collected, the corresponding weak pointer automatically becomes nil
7. 7 References from GC's Perspective 1. GC "doesn't see" weak references: Ignored in reachability analysis (objects don't survive with only weak references) 2. Value() invalidation: When the target becomes eligible for collection, Value() method may return nil
 3. Non-deterministic timing: When it starts returning nil depends on GC
8. 8 Why Do We Need Weak References? B, C are unnecessary but not freed → 200MB of wasted memory
9. 9 Introducing Weak References B, C are collected → Only necessary memory is used
10. 10 Typical Use Cases 1. Self-cleaning Cache: Unnecessary data is automatically collected by GC By using weak pointers, data is automatically GC'd when no longer needed cache.Store(key, weak.Make(data)) if wp := cache.Load(key); wp.Value() != nil { return wp.Value() // Still alive }
11. 11 Typical Use Cases 2. Canonicalization / Deduplication: The technology underlying the unique package ip.z = unique.Make(zoneDetail{zone: "eth0"}) // Same value → same Handle, fast comparison • The purpose of canonicalization (interning) is to consolidate multiple identical values into a single canonical copy • This leads to memory savings
12. 12 Typical Use Cases 3. Lifetime Coupling: Properly managing lifetimes between multiple objects • When a listener becomes unnecessary elsewhere (no strong references remain), it's automatically removed from the bus • Efficiently manages memory whose lifetime is tied to another object's lifetime type EventBus struct { listeners []weak.Pointer[Listener] }
13. 13 The Journey to Weak References
14. 14 Early Proposal(2010) ref:https://groups.google.com/g/golang-nuts/c/9br5GgucQcU
15. 15 Turning Point: unique Package Implementation (2023) • Need to implement canonicalization in the unique package ◦ Identical values share the same memory ◦ Mechanism that doesn't prevent garbage collection of unused entries • Existing workarounds (borrowed from go4.org/intern) had problems
16. 16 Lessons from go4.org/intern • Requires 3 garbage collection cycles • Depends on unsafe operations • Relies on unpublished collector behavior // go4.org/intern implementation (using unsafe) type Value struct { cmpVal interface{} resurrected bool // Resurrection flag } var ( mu sync.Mutex valMap = map[key]uintptr{} // Hide *Value as uintptr ) func get(k key) *Value { if addr, ok := valMap[k]; ok { v = (*Value)((unsafe.Pointer)(addr)) // unsafe conversion v.resurrected = true // Mark as resurrected } // ... }
17. 17 Lessons from go4.org/intern Problems • Requires 3 garbage collection cycles • Depends on unsafe operations • Relies on unpublished collector behavior
18. 18 "Unsafe string interning" in Go by Matt Layher > The unsafe techniques used in go4.org/intern are incredibly subtle and have some seriously sharp edges. There have been several different iterations of this package written by some of the most experienced Go programmers on the planet that still managed to have subtle defects and data races ref: https://mdlayher.com/blog/unsafe-string-interning-in-go/
19. 19 Michael Knyszek's Discovery > "This was discovered during the implementation of unique package and it was compelling enough that it led to this proposal." — issue #67552 • The need for weak references became clear during unique package implementation • For efficient management of canonicalization maps • The weak reference implementation was simple and compelling, leading to this proposal
20. 20 Simple API Design 
 
 
 • Make: Creates a weak reference • Value: Gets a strong reference (*T) (may be nil) type Pointer func Make[T any](ptr *T) Pointer[T] func (p Pointer[T]) Value() *T
21. 21 Design Innovations in Internal Implementation
22. 22 1. Avoiding Object Resurrection What is Object Resurrection? ◦ A phenomenon where an object that should be collected by GC becomes accessible again during finalizer execution
23. 23 1. Avoiding Object Resurrection What is Object Resurrection? ◦ A phenomenon where an object that should be collected by GC becomes accessible again during finalizer execution
24. 24 1. Avoiding Object Resurrection Problems When Resurrection is Allowed ◦ 2 GC cycles required: First for resurrection check, second for actual collection ◦ Permanent leak with circular references: Resurrected objects in cycles never get collected ◦ Race conditions: Objects retrieved from weak references may resurrect mid-operation, causing inconsistent internal state
25. 25 Short Weak References vs Long Weak References Short Weak References (Adopted by Go) Becomes nil before finalizer execution 1. obj becomes unreachable → 2. wp.Value() returns nil → 3. Finalizer executes obj := &MyObject{} wp := weak.Make(obj) runtime.SetFinalizer(obj, cleanup)
26. 26 Short Weak References vs Long Weak References Long Weak References (Not adopted) Tracks object even after finalizer execution 1. obj becomes unreachable → 2. Finalizer executes → 3. nil after resurrection check // Hypothetical implementation obj := &MyObject{} wp := longWeak.Make(obj) // Hypothetical API runtime.SetFinalizer(obj, func(o *MyObject) { resurrect(o) // Resurrect it })
27. 27 Why Short Weak References Were Chosen • Avoids resurrection races: Safe even if resurrection occurs since weak ref is already nil • Implementation simplicity: No complex resurrection tracking logic needed • Proven in other languages: C# default, Java standard - proven sufficient in both • Performance: Completes in 1 GC cycle, no additional resurrection checks needed
29. 29 2. Preventing Dangling References • What is a Dangling Reference? ◦ A pointer pointing to already freed memory ◦ Accessing it can cause crashes or unpredictable behavior ◦ With weak references, the referenced memory can be collected at any time, creating danger of dangling references with direct access like regular pointers
30. 30 2. Preventing Dangling References • What is a Dangling Reference? ◦ A pointer pointing to already freed memory ◦ Accessing it can cause crashes or unpredictable behavior ◦ With weak references, the referenced memory can be collected at any time, creating danger of dangling references with direct access like regular pointers
31. 31 Solution Innovations • Automatic nil-ing: Weak pointers never touch freed memory. They first read a runtime-managed handle; once that handle is zero, `Value()` simply returns `nil`. • Indirection via word-sized handle: Weak pointers share a word-sized (8-byte) handle instead of pointing to the object itself. • Atomic, batched invalidation: All weak pointers to the same `(object, offset)` share the handle, so a single atomic zero invalidates them simultaneously
32. 32 Quick GC Review: How Go's Tracing GC Works GC Cycle 1. Marking 2. Mark Termination 3. Sweep
33. 33 Invalidation at Mark Termination GC Cycle 1. Marking └─ Weak references don't keep objects alive 2. Mark Termination └─ Value() blocks conversions until the relevant span is swept. 3. Sweep └─The runtime zeroes the shared handle for unreachable objects (this invalidates all weak pointers to those objects). 4. Observation └─ After sweep completes, Value() returns nil. If Value() is called right after mark termination, it will first force sweep of the relevant span, then read the handle (→ nil).
34. 34 Architecture Structure
35. 35 ①Creating the 8-Byte Indirection // From src/runtime/mheap.go: getOrAddWeakHandle() func getOrAddWeakHandle(p unsafe.Pointer) *atomic.Uintptr { // Check for existing handle first... if handle := getWeakHandle(p); handle != nil { return handle } // Allocate the special record (metadata) lock(&mheap_.speciallock) s := (*specialWeakHandle)(mheap_.specialWeakHandleAlloc.alloc()) unlock(&mheap_.speciallock) // Create the 8-byte indirection object (the critical piece!) handle := new(atomic.Uintptr) // ← Heap-allocated shared handle s.special.kind = _KindSpecialWeakHandle s.handle = handle // ← Link special → handle handle.Store(uintptr(p)) // ← Store target address
36. 36 // Attach to the object's span if addspecial(p, &s.special, false) { // If GC is running, mark the handle itself to keep it alive if gcphase != _GCoff { scanblock(uintptr(unsafe.Pointer(&s.handle)), goarch.PtrSize, &oneptrmask[0], gcw, nil) } return handle } // ... handle race condition ... } Result: One `atomic.Uintptr` per target, shared by all weak pointers
37. 37 ②The One-Shot Invalidation: When GC collects the target // From src/runtime/mheap.go: freeSpecial() func freeSpecial(s *special, p unsafe.Pointer, size uintptr) { switch s.kind { // ... other special types ... case _KindSpecialWeakHandle: sw := (*specialWeakHandle)(unsafe.Pointer(s)) // THE CRITICAL LINE: One atomic write invalidates ALL weak pointers sw.handle.Store(0) // ← 0 means "target was collected" lock(&mheap_.speciallock) mheap_.specialWeakHandleAlloc.free(unsafe.Pointer(s)) unlock(&mheap_.speciallock) // ... } }
38. 38 ②The One-Shot Invalidation: Reading through the indirection (from weak package) // From src/weak/pointer.go type Pointer[T any] struct { _ [0]*T u unsafe.Pointer // ← Points to the indirection handle } func (p Pointer[T]) Value() *T { if p.u == nil { return nil // ← Zero value or nil pointer case } // Delegate to runtime for safe weak→strong conversion return (*T)(runtime_makeStrongFromWeak(p.u)) }
39. 39 ②The One-Shot Invalidation: Reading through the indirection (from weak package) // From runtime/mheap.go func internal_weak_runtime_makeStrongFromWeak(u unsafe.Pointer) unsafe.Pointer { handle := (*atomic.Uintptr)(u) // ← Cast to handle type mp := acquirem() if work.strongFromWeak.block { releasem(mp) mp = gcParkStrongFromWeak() // ← Wait for mark to complete } p := handle.Load() // ← Atomic read from shared indirection if p == 0 { releasem(mp) return nil // ← Object was collected }
40. 40 span := spanOfHeap(p) if span == nil { if isGoPointerWithoutSpan(unsafe.Pointer(p)) { releasem(mp) return unsafe.Pointer(p) // ← Immortal object } releasem(mp) return nil } span.ensureSwept() // ← Synchronize with GC ptr := unsafe.Pointer(handle.Load()) // ← Re-read after sweep if gcphase != _GCoff { shade(uintptr(ptr)) // ← Mark target if GC running } releasem(mp) return ptr }
41. 41 ③Safe Weak-to-Strong Conversion // From runtime/mheap.go: internal_weak_runtime_makeStrongFromWeak() func internal_weak_runtime_makeStrongFromWeak(u unsafe.Pointer) unsafe.Pointer { handle := (*atomic.Uintptr)(u) mp := acquirem() // Prevent preemption // CRITICAL: Wait if GC is marking (prevents resurrection race) if work.strongFromWeak.block { releasem(mp) mp = gcParkStrongFromWeak() // ← Park until mark completes } p := handle.Load() // Read through indirection if p == 0 { releasem(mp) return nil // Already collected } Preventing races during GC:
42. 42 span := spanOfHeap(p) if span == nil { // Handle immortal objects... return unsafe.Pointer(p) } // CRITICAL: Ensure span is swept before trusting the pointer span.ensureSwept() // ← Synchronize with GC ptr := unsafe.Pointer(handle.Load()) // Re-read after sweep // If GC is running, mark the target if gcphase != _GCoff { shade(uintptr(ptr)) // ← Keep target alive this cycle } releasem(mp) return ptr } Three safety mechanisms: Block during mark → Ensure sweep → Mark if necessary
44. 44 3. Resolving GC Abstraction Breakage and Difficulty of Use What is "GC Abstraction Breakage"? • Go's ideal: Developers don't think about memory management ("infinite memory" abstraction) • Weak reference problem: Suddenly "when GC ran" becomes visible
45. 45 Specific Concerns • Unpredictability: Same code may produce different results each execution • Loss of composability: Difficult to combine functions (may become nil midway)
46. 46 Solution Approach • Simple API design: Simple API with only `Make` and `Value` • Clear guidance: Document when to use and when to avoid (see [GC Guide](https://golang.org/doc/gc-guide#Finalizers _cleanups_and_weak_pointers)) • Learn from other languages: Reference patterns from Java, C#, Python, etc.
47. 47 Comparison with Other Languages Feature Go Java C# GC Type Tracing Generational tracing (common, varies) Generational tracing GC Weak ref representation weak.Pointer[T].Va lue()
 WeakReference<T>.g et() + Queue WeakReference.Targ et
 Invalidation timing Unreachable → may become nil at finalizer queue Unreachable → nil before finalizer queue Unreachable → null when finalizable Short: null when finalizable Long: null after finalizer Resurrection possibility Prevented with "short" weak refs Prevented with weak refs Possible in finalizers
48. 48 Implementation Differences // Go: Cooperates with tracing GC wp := weak.Make(&obj) // GC determines obj unreachable → wp.Value() may return nil // C#: Choice of short or long weak references WeakReference shortWr = new WeakReference(obj); // null when finalizable WeakReference longWr = new WeakReference(obj, true); // tracks resurrection // Finalizer can resurrect; long ref still valid after resurrection Go's design choice: Weak references that truly prevent resurrection
49. 49 Practical Patterns and Caveats
50. 50 Pattern 1: Cache Implementation type WeakCache[K comparable, V any] struct { m sync.Map // Stores weak.Pointer[V] } func (c *WeakCache[K, V]) Set(key K, value *V) { wp := weak.Make(value) c.m.Store(key, wp) // Set auto-deletion with AddCleanup runtime.AddCleanup(value, func(k K) { if v, ok := c.m.Load(k); ok { if v.(weak.Pointer[V]) == wp { c.m.Delete(k) } } }, key) } func (c *WeakCache[K, V]) Get(key K) (*V, bool) { if v, ok := c.m.Load(key); ok { if ptr := v.(weak.Pointer[V]).Value(); ptr != nil { return ptr, true // Object exists } c.m.Delete(key) // Delete if GC'd } return nil, false }
51. 51 Implementation Key Points 1. weak.Make(): Create weak reference 2. weak.Value(): nil check is essential 3. runtime.AddCleanup: Auto-deletion of entries
52. 52 AddCleanup: Role and Usage What does it do? • Registers a lightweight, GC-coupled cleanup that runs at most once for a target’s lifetime • Does not receive the target object (you pass only minimal context like keys/IDs) • Does not keep the target alive and cannot resurrect it How it differs from finalizers (design intent) • Focused on cleaning up external/auxiliary structures (e.g., removing from a map) • Cleanup does not receive the target object, preventing resurrection • Intended to be short and non-blocking (avoid heavy I/O or long-held locks)
53. 53 AddCleanup Best practices/Use Cases 1. Do not capture the target in closures (pass only necessary info like a key) 2. Keep cleanup short and non-blocking 3. Leverage that weak pointers are comparable; confirm identity before deletion // Safely delete only the entry associated with a value that became unreachable runtime.AddCleanup(value, func(k K) { if v, ok := c.m.Load(k); ok { if v.(weak.Pointer[V]) == wp { // Weak pointers remain comparable c.m.Delete(k) } } }, key) Common use cases • Auto-deletion of cache/side-map entries • Removing external handles or registered listeners, coupled to the target's lifetime
54. 54 Pattern 2: String Interning (unique) // Traditional: String content comparison func slowCompare(s1, s2 string) bool { return s1 == s2 // O(n) - proportional to string length } // Using unique: Handle comparison (constant time) func fastCompare(h1, h2 unique.Handle[string]) bool { return h1 == h2 } Speeding up with Pointer Comparison
55. 55 Pattern 3: Managing Lifetime Coupling type EventBus struct { listeners []weak.Pointer[Listener] mu sync.RWMutex } type Listener struct { onEvent func(Event) // other fields... } func (bus *EventBus) Subscribe(listener *Listener) { bus.mu.Lock() defer bus.mu.Unlock() bus.listeners = append(bus.listeners, weak.Make(listener)) }
56. 56 Pattern 3: Managing Lifetime Coupling func (bus *EventBus) Publish(event Event) { bus.mu.RLock() current := make([]weak.Pointer[Listener], len(bus.listeners)) copy(current, bus.listeners) bus.mu.RUnlock() // Process only valid listeners and cleanup simultaneously validListeners := make([]weak.Pointer[Listener], 0, len(current)) for _, wp := range current { if listener := wp.Value(); listener != nil { listener.onEvent(event) validListeners = append(validListeners, wp) } } // Remove invalid references bus.mu.Lock() bus.listeners = validListeners bus.mu.Unlock() }
57. 57 Actual Effects and Benefits • Automatic memory management: Objects no longer needed are automatically deleted • Lifetime coupling: Properly manages lifetimes of related objects • GC hints: Notifies GC that "it's safe to delete this resource"
58. 58 Caveats Tiny Object Issue // Be careful with very small sizes that don't contain pointers type Tiny struct { a, b int32 // Very small & doesn't contain pointers } // Example workaround: Increase size type NotTiny struct { a, b int32 _ [9]byte // Padding to 17+ bytes }
59. 59 Caveats Indirection Object Overhead Memory Cost of Weak References Every unique weak reference target requires an 8-byte indirection object pointer 8-byte handle target Weak Pointer Indirection
60. 60 Caveats // Dangerous code data := wp.Value() process(data) // May panic on nil // Safe code if data := wp.Value(); data != nil { process(data) } Importance of nil Checks
61. 61 Summary 1. Achieving Automatic Memory Management • Cache's unnecessary data naturally freed • No need for unsafe workarounds from go4.org/intern 2. Simple and Safe API • Create with Make(), retrieve with Value() (nil check required) • Reliable cleanup with runtime.AddCleanup
 3. Practical Applications • Cache, string interning (unique), event listener management
62. 62 Weak references are a new option for "memory-aware design"
63. 63 : kuro(naoki kuroda) @knkurokuro7