Zekun Li - There and Back Again: Disable and re-enable garbage collector at Instagram

Python's cyclic garbage collector wonderfully hides the complexity of memory management from the programmer. But we pay the price in performance. Ever wondered how that works? In this talk, you'll learn how garbage collection is designed in Python, what the tradeoffs are and how Instagram battled copy-on-write memory issues by disabling the garbage collector entirely.

You'll also learn why that isn't such a great idea after all and how we ended up extending the garbage collector API which allowed us to (mostly) re-enable garbage collection. We'll discuss our upstream contributions to the garbage collector that landed in Python 3.6 and 3.7.

This is an in-depth talk about memory management but no prior experience with CPython internals is necessary to follow it.

https://us.pycon.org/2018/schedule/presentation/72/

Transcript

1. THERE AND BACK AGAIN: GARBAGE COLLECTION AT INSTAGRAM Zekun Li 李李泽昆 Instagram Infrastructure zekun@fb.com
2. AGENDA 1 Background 2 Why and How Instagram disable GC 3 Why and How Instagram re-enable GC, mostly
3. BACKGROUND
4. MEMORY MANAGEMENT IN PYTHON 4 Reference Count Garbage Collector for cyclic reference
5. MEMORY MANAGEMENT IN PYTHON 5 foo = [] bar = [] __main__ bar foo del foo del bar
6. MEMORY MANAGEMENT IN PYTHON 6 foo = [] bar = [] foo.append(bar) bar.append(foo) __main__ bar foo del foo del bar
7. Process 2 Memory Page Memory Page COPY-ON-WRITE SEMANTIC 7 Process 1 World Hello
8. INSTAGRAM TECH STACK 8
9. DISABLE GARBAGE COLLECTION
10. WHY DISABLE GC? 10
11. OBSERVATION 11 500 MB 350 MB 150 MB 150 MB 150 MB 150 MB 150 MB 150 MB
13. WHERE'S MY SHARED MEMORY? 13
14. PROFILING 14
15. WHY GARBAGE COLLECTION? 15 /* GC information is stored BEFORE the object structure. */ typedef union _gc_head { struct { union _gc_head *gc_next; union _gc_head *gc_prev; Py_ssize_t gc_refs; } gc; double dummy; /* force worst-case alignment */ } PyGC_Head;
16. Memory Page Memory Page 16 PyGC_HEAD PyObject
17. 17 CAN WE DISABLE THE GC? YES, WE HAVE REFERENCE COUNT
18. HOW TO DISABLE GC? 18
19. DISABLE GC 19 gc.disable() atexit.register(os._exit, 0) gc.set_threshold(0) # fixed in python 3.6
20. RESULTS Memory util -15% Shared memory +100mb Instruction/cycle +10%
21. RE-ENABLE GARBAGE COLLECTION
22. WHY DISABLE GC IS BAD? 22
23. 600M 1200M 0 3000 AFTER ONE YEAR 23
24. 24 HOW TO MAKE GC COW FRIENDLY?
25. 25 REDESIGN PYGC_HEAD
26. REDESIGN GC HEAD 26 /* GC information is stored BEFORE the object structure. */ typedef union _gc_head { struct { union _gc_head *gc_next; union _gc_head *gc_prev; Py_ssize_t gc_refs; } gc; double dummy; /* force worst-case alignment */ } PyGC_Head;
27. REDESIGN GC HEAD 27 /* GC information is stored BEFORE the object structure. */ typedef union _gc_head { struct { union _gc_head *head; } gc_ptr; double dummy; /* force worst-case alignment */ } PyGC_Head_Ptr;
28. Memory Page Memory Page Memory Page Memory Page Memory Page REDESIGN GC HEAD 28 PyGC_HEAD PyObject PyGC_HEAD_Ptr
29. WORKING PROOF 29 lists = [] strs = [] for i in range(16000): lists.append([]) for j in range(40): strs.append(' ' * 8) ~60MB ~0.9MB
30. RESULTS
31. POINTERS TAKE SPACE 31 Memory Page Memory Page 16 BYTES 1,000,000 OBJECTS 80 PROCESSES ~1 GB
32. 32 FREEZE OBJECTS
33. 33 FREEZE OBJECTS static PyObject * gc_freeze(PyObject *module) { for (int i = 0; i < NUM_GENERATIONS; ++i) { gc_list_merge(GEN_HEAD(i), &permanent_generation.head); generations[i].count = 0; } Py_RETURN_NONE; } gc.freeze() # upstream to python 3.7!
34. 34 NO GC BEFORE FORK?
35. Memory Page 35 MEMORY POOL Memory Page PyGC_HEAD PyObject Collection Memory Page Fork Memory Page CoW!!
36. RESULTS GC Enabled GC Disabled
37. WRAP UP 1 Disable garbage collection to avoid copy-on-write 2 Re-enable garbage collection by freezing objects 3 No gc before fork to avoid copy-on-write by re-allocation
38. https://instagram-engineering.com