ITERATORS+GENERATORS: THE ROAD TO VIRTUAL INFINITE RAM

Transcript

1. Alexandre Daubois - Dutch PHP Conference 2024 - Amsterdam The Road To Virtual Infinite RAM Iterators & Generators
2. Who am I?
3. Alexandre Daubois (@alexdaubois) Symfony Lead Developer at WanadevDigital, France One of the main contributors to Symfony (code+docs) Conference speaker, blogpost and book author (7th time in the Netherlands 🇳🇱)
6. 10 000 000 Projects created
8. Everything is part of one big JSON collection. You said « collection »?
9. A B C D E A container of n elements, which could be stored...
10. A container of n elements, which could be stored...
11. Stored collection order Desired traversal order
12. Stored collection order Desired traversal order
13. Stored collection order Desired traversal order
14. What if you can just do this with « foreach » ?
15. Let me show you.
16. The Iterator Design Pattern When a simple « array access » is not enough
19. Iterators define how a collection should be browsed
20. Here is an iterator.
22. Inner structure is hidden
23. Access logic decoupled from storage
24. The collection actually only stores data
25. Create a new traversal behavior at light speed
26. Easy to test.
27. PHP got your back.
29. AppendIterator ArrayIterator CachingIterator CallbackFilterIterator DirectoryIterator EmptyIterator FilesystemIterator FilterIterator GlobIterator In fi niteIterator IteratorIterator LimitIterator MultipleIterator NoRewindIterator ParentIterator RecursiveArrayIterator RecursiveCachingIterator RecursiveCallbackFilterIterator RecursiveDirectoryIterator RecursiveFilterIterator RecursiveIteratorIterator RecursiveRegexIterator RecursiveTreeIterator RegexIterator
30. Let me show you.
31. Let’s talk about Virtual Infinite RAM
32. php -d memory_limit=-1
33. Yield
34. Generators are the answer. It’s all about compromises.
35. What if the data source is… 1 billion elements in a remote cache? A fi le of 1 TB? A streamed video? An HTTP response of 800MB?
36. Good luck with your RAM 👋
37. Generators create values on the fly 🪽
38. Let’s see how?
39. Generators are iterators.
40. Delegated generation
41. Delegated generation + IteratorAggregate
42. Generators are flexible.
43. 1. Identify your bottleneck(s)
44. 2. I/O vs Memory vs Network
45. 3. Create the good generator
46. PERFORMANCE
47. Benchmarking range() By Nikita Popov
48. Three implementations Native implementation Iterator implementation Generator implementation https://gist.github.com/nikic/2975796
49. 40 % Faster than the native implementation Generators are up to
50. 400 % Faster than the iterator implementation Generators are up to
51. Nikita Popov, Generators RFC For small ranges (around one hundred elements), [...] generators are slightly slower than the native implementation, but still faster than the iterator variant.
52. How do they work?
54. Diving into PHP internals
55. Zend Generator Instructions of the generating function Execution context Local variables Current instruction
56. No generated value is stored: garbage collector can do its job
58. In 3rd party libraries
59. Symfony Component Example of usage Console Manage the output of your command (real-time output + preserves memory) Finder Discover directories and fi les lazily (preserves I/O) HttpFoundation Streaming responses (preserves network, memory and I/O) HttpClient Responses multiplexing (async) Process Fetch running process output (realtime output + preserves memory)
60. Doctrine
61. PHPUnit
62. LazyStream
63. Tomorrow in yours!
64. Thank you 🇳🇱 @alexdaubois