Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Hunting Down Memory Leaks with php Meminfo

Hunting Down Memory Leaks with php Meminfo

To better understand how a memory leak can occurs in PHP, this presentation intr
oduces how memory freeing is managed in PHP. Then it shows how the php Meminfo e
xtension can be used to debug and to help fix memory leaks.

Benoit Jacquemont

May 10, 2019
Tweet

More Decks by Benoit Jacquemont

Other Decks in Programming

Transcript

  1. HUNTING DOWN HUNTING DOWN HUNTING DOWN MEMORY LEAKS WITH MEMORY

    LEAKS WITH MEMORY LEAKS WITH PHP MEMINFO PHP MEMINFO PHP MEMINFO Benoit Jacquemont @bjacquemont
  2. What Is A Memory Leak? A memory leak occurs when

    memory which is no longer needed is not released. en.wikipedia.org/wiki/Memory_leak
  3. Memory Leaks: Should You Care? Usually, PHP process life time

    = HTTP exchange lifetime PHP process memory released a er each HTTP exchange Yes but... more and more capable-but-complex stacks long running background jobs
  4. Why I Care About Memory Leaks E-COMMERCE MOBILE APPLICATION PRINT

    CATALOG POINTS OF SALE ERP MEDIA SERVER SUPPLIERS PURCHASING DPT MARKETING DPT CSV FTP XML XLS SUPPLIERS PORTAL ENRIC H TRAN SLATE CO N T RO L Import Jobs Export Jobs Mass Edition Jobs Rules Engine Jobs ...
  5. The Effects Of A Memory Leak Less memory available for

    other processes (surprise !) The more memory your program uses, the slower it becomes
  6. 1st object will be released only when 2nd object is

    released 2nd object will be released only when 1st object is released Neither of them can be released by the refcounter... The Refcounter Cannot Handle Circular References
  7. "Yeah sure, but circular references don't exist in the real

    world..." Tree structure: parent → children → parent Doctrine bidirectional mapping: entity1 → entity2 → entity1 ...
  8. But my memory usage keeps increasing! function buildObjects() { $objA

    = new StdClass(); $objB = new StdClass(); $objA->attr = $objB; $objB->attr = $objA; } for ($i = 0; $i < 2000; $i++) { buildObjects(); }
  9. Keep Calm And Wait For The Collect! There's a delay

    before the Collector cleans up memory
  10. Circular Reference Collection How It Works Each time an object

    or array has its refcount decremented, it gets added to the collector buffer.
  11. Memory Leak = Items Non Needed Anymore But Still Referenced,

    Directly Or Indirectly, By A Variable Still Alive
  12. Let's Leak ! function buildObjects() { $objA = new StdClass();

    $objB = new StdClass(); $objA->attr = $objB; $objB->attr = $objA; return $objA; } $leakHolder = []; for ($i = 0; $i < 200000; $i++) { $object = buildObjects(); if ($i % 10 === 0) { $leakHolder[] = $object; } }
  13. Why The Slow Down When The GC Buffer Is Full

    Of Referenced Items? Each time a refcount is decremented, your program is interrupted to rescan the buffer!
  14. Hunting Memory Leaks With Profilers only provides information on when

    (at function level) memory is taken or released not on what or why
  15. PHP Meminfo MIT License Two parts: the extension itself, to

    dump memory content analyzers to work with memory dump files github.com/BitOne/php-meminfo
  16. Import Example foreach ($fileContent as $data) { $counter++; $user =

    new User(); $user->setLogin($data[0]); $user->setEmail($data[1]); $user->setFullName($data[2]); $this->getEntityManager()->persist($user); if (0 === $counter % 1000) { $this->getEntityManager()->flush(); } } $this->getEntityManager()->flush();
  17. Dumping Memory Content ... $this->getEntityManager()->persist($user); if (0 === $counter %

    1000) { $this->getEntityManager()->flush(); meminfo_dump(fopen("mem$counter.json", 'w')); } ...
  18. Memory Content Summary $ bin/analyzer summary mem1000.json +-----------------------+-----------------+---------------------+ | Type

    | Instances Count | Cumulated Self Size | +-----------------------+-----------------+---------------------+ | string | 7641 | 303612 | | integer | 4418 | 70688 | | array | 2691 | 193752 | | AppBundle\Entity\User | 1001 | 72072 | | boolean | 463 | 7408 | | null | 413 | 6608 | ... $ bin/analyzer summary mem5000.json +-----------------------+-----------------+---------------------+ | Type | Instances Count | Cumulated Self Size | +-----------------------+-----------------+---------------------+ | string | 31638 | 1072852 | | integer | 20418 | 326688 | | array | 10690 | 769680 | | AppBundle\Entity\User | 5001 | 360072 | | boolean | 463 | 7408 |
  19. Finding A Good Leak Candidate $ bin/analyzer query -f "class~User"

    -v mem5000.json +-----------+------------------------------+---------------------+ | Item ids | Item data | Children | +-----------+------------------------------+---------------------+ | 0x2b46aa8 | Type: object | id: 0x2b3fa48 | | | Class: AppBundle\Entity\User | login: 0x2af8440 | | | Object Handle: 374 | fullName: 0x2584c48 | | | Size: 56 B | email: 0x2592580 | | | Is root: No | | +-----------+------------------------------+---------------------+ | 0x2b489f0 | Type: object | id: 0x2b47ae8 | | | Class: AppBundle\Entity\User | login: 0x2b45738 | | | Object Handle: 376 | fullName: 0x2a25448 | | | Size: 56 B | email: 0x2884590 | | | Is root: No | | +-----------+------------------------------+---------------------+ | 0x2b49250 | Type: object | id: 0x2acb7a0 | | | Class: AppBundle\Entity\User | login: 0x2b45df8 | | | Object Handle: 377 | fullName: 0x2883170 | | | Size: 56 B | email: 0x2abd7f0 | | | Is root: No | |
  20. Why Is This Object Still In Memory? $ bin/analyzer ref-path

    0x2b46aa8 mem5000.json Found 1 path(s) Path to 0x2b46aa8 (<GLOBAL>)$GLOBALS["kernel"] ->container ->services["doctrine.orm.default_entity_manager"] ->unitOfWork ->identityMap["AppBundle\Entity\User"]["9"]
  21. Fixing The Leak! .... $this->getEntityManager()->persist($user); if (0 === $counter %

    1000) { $this->getEntityManager()->flush(); $this->getEntityManager()->clear('AppBundle\\Entity\\User'); } ...
  22. Memory Leak - Good Practices monitor your long running processes

    speed and memory usage avoid (or at least be aware of) stateful services use a reasonable memory limit (not -1 !) keep the number of items in memory at a low level