Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scaling PHP to 40 Million Uniques

Scaling PHP to 40 Million Uniques

This is a talk I gave at Northeast PHP 2013 in Boston, MA.

Links: jkle.in/nephp

Jonathan Klein

August 17, 2013
Tweet

More Decks by Jonathan Klein

Other Decks in Technology

Transcript

  1. Scaling PHP to 40 Million Uniques Northeast PHP 2013 Jonathan

    Klein @jonathanklein Tuesday, September 24, 13
  2. Scaling PHP to 40 Million Uniques Northeast PHP 2013 Jonathan

    Klein @jonathanklein Tuesday, September 24, 13
  3. Scaling PHP to 60 Million Uniques Northeast PHP 2013 Jonathan

    Klein @jonathanklein Tuesday, September 24, 13
  4. Who Am I? • Performance Engineer at Etsy • Boston

    Web Perf Meetup Organizer Tuesday, September 24, 13
  5. Who Am I? • Performance Engineer at Etsy • Boston

    Web Perf Meetup Organizer • Previously at Wayfair Tuesday, September 24, 13
  6. Who Am I? • Performance Engineer at Etsy • Boston

    Web Perf Meetup Organizer • Previously at Wayfair • Led team that converted to PHP Tuesday, September 24, 13
  7. Some Etsy Stats • 1.5 billion page views/month • Almost

    $1B in sales last year Tuesday, September 24, 13
  8. Some Etsy Stats • 1.5 billion page views/month • Almost

    $1B in sales last year • Over 1M lines of PHP Tuesday, September 24, 13
  9. Some Etsy Stats • 1.5 billion page views/month • Almost

    $1B in sales last year • Over 1M lines of PHP • 60M+ unique visitors/month Tuesday, September 24, 13
  10. A few others... • Search (Solr) • Gearman • Redis

    • Postgres (legacy) Tuesday, September 24, 13
  11. Apache • Apache 2.2 • Prefork MPM • mod_php5 StartServers

    30 MinSpareServers 30 MaxSpareServers 60 ServerLimit 60 MaxClients 60 MaxRequestsPerChild 0 Tuesday, September 24, 13
  12. PHP • PHP 5.4 • Zend OPCache (3GB Memory Segment)

    • memory_limit: 128M Tuesday, September 24, 13
  13. PHP • PHP 5.4 • Zend OPCache (3GB Memory Segment)

    • memory_limit: 128M • max_execution_time: 30 Tuesday, September 24, 13
  14. Optimizing PHP • See last year’s talk • Use an

    Opcode Cache http://www.slideshare.net/jnklein/northeast-php-high-performance-php Tuesday, September 24, 13
  15. Optimizing PHP • See last year’s talk • Use an

    Opcode Cache • xhprof http://www.slideshare.net/jnklein/northeast-php-high-performance-php Tuesday, September 24, 13
  16. Optimizing PHP • See last year’s talk • Use an

    Opcode Cache • xhprof • StatsD/Graphite http://www.slideshare.net/jnklein/northeast-php-high-performance-php Tuesday, September 24, 13
  17. Optimizing PHP • See last year’s talk • Use an

    Opcode Cache • xhprof • StatsD/Graphite • Find hotspots http://www.slideshare.net/jnklein/northeast-php-high-performance-php Tuesday, September 24, 13
  18. Optimizing PHP • See last year’s talk • Use an

    Opcode Cache • xhprof • StatsD/Graphite • Find hotspots • Upgrade it http://www.slideshare.net/jnklein/northeast-php-high-performance-php Tuesday, September 24, 13
  19. Translations class language_de { static $translations = array( "<hash>" =>

    array("content" => 'Accessoires', "file" => "TAXONOMY"), "<hash>" => array("content" => 'Keramik', "file" => "CATEGORY"), "<hash>" => array("content" => 'Dekorieren', "file" => "CATEGORY"), "<hash>" => array("content" => 'Getaggt %s', "file" => "Foo.php"), "<hash>" => array("content" => 'Badezusatz', "file" => "CATEGORY"), "<hash>" => array("content" => 'Datum:{% $date %}', "file" => "bar.tpl") ... snip ... ); } Tuesday, September 24, 13
  20. Deploying PHP • mod_realdoc - Apache module • incpath -

    PHP Extension http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/ Tuesday, September 24, 13
  21. Deploying PHP • mod_realdoc - Apache module • incpath -

    PHP Extension • A/B symlink swap http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/ Tuesday, September 24, 13
  22. Deploying PHP • mod_realdoc - Apache module • incpath -

    PHP Extension • A/B symlink swap • Avoid recompilation to opcodes http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/ Tuesday, September 24, 13
  23. PHP 5.4 vs. 5.3 - CPU PHP 5.3 PHP 5.4

    Tuesday, September 24, 13
  24. PHP 5.4 vs. 5.3 - Memory PHP 5.3 PHP 5.4

    Tuesday, September 24, 13
  25. Memcached • Roughly same hardware as webs • More memory

    (48GB) • Shard keys across servers Tuesday, September 24, 13
  26. Memcached • Roughly same hardware as webs • More memory

    (48GB) • Shard keys across servers • mctop - https://github.com/etsy/mctop Tuesday, September 24, 13
  27. MySQL • MySQL 5.5 (http://jkle.in/nephp) • ~20 master/master pairs •

    innodb • thread_cache=800 Tuesday, September 24, 13
  28. MySQL • MySQL 5.5 (http://jkle.in/nephp) • ~20 master/master pairs •

    innodb • thread_cache=800 • max_connections=2500 Tuesday, September 24, 13
  29. MySQL • MySQL 5.5 (http://jkle.in/nephp) • ~20 master/master pairs •

    innodb • thread_cache=800 • max_connections=2500 • [Almost] no joins Tuesday, September 24, 13
  30. Offloading Requests • Multiple CDNs • Img hit rate >

    90% • CSS/JS hit rate ~100% Tuesday, September 24, 13
  31. Offloading Requests • Multiple CDNs • Img hit rate >

    90% • CSS/JS hit rate ~100% • Edge serves ~7000 reqs/sec Tuesday, September 24, 13
  32. // Owner: Username $server_config['foo']['bar'] = array( 'users' => array('jsmith', 'jdoe'),

    ); // Owner: Username $server_config['baz'] = [ 'enabled' => [ 'variant1' => 0, 'variant2' => 0, 'variant3' => 50 ], ]; // Owner: Username $server_config['qux'] = array( 'enabled' => 100, 'data' => array( 'somedata' ) ); Tuesday, September 24, 13
  33. Options • Scale Horizontally (more shards) • Scale Vertically (SSD,

    bigger drives) • Change architecture Tuesday, September 24, 13
  34. Architecture Changes • Ongoing conversation • Separate logical/physical shards •

    Complex • Server failure • Migrating data Tuesday, September 24, 13
  35. Architecture Changes • Ongoing conversation • Separate logical/physical shards •

    Complex • Server failure • Migrating data • Consistency Tuesday, September 24, 13