Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scaling PHP to 40 Million Uniques

Sponsored · SiteGround - Reliable hosting with speed, security, and support you can count on. →
Avatar for Jonathan Klein Jonathan Klein
August 17, 2013
Technology
1.3k
3

Scaling PHP to 40 Million Uniques

This is a talk I gave at Northeast PHP 2013 in Boston, MA.

Links: jkle.in/nephp

Avatar for Jonathan Klein

Jonathan Klein

August 17, 2013

More Decks by Jonathan Klein

See All by Jonathan Klein
Cognitive Biases in Engineering Organizations - Craft Conf
Avatar for Jonathan Klein jklein
15
510
Demystifying SPDY and HTTP/2
Avatar for Jonathan Klein jklein
1
690
Demystifying SPDY and HTTP/2
Avatar for Jonathan Klein jklein
2
1.2k
Cognitive Biases in Engineering Organizations
Avatar for Jonathan Klein jklein
4
2k
Composer: From Beginner to Expert
Avatar for Jonathan Klein jklein
1
1.1k
Upgrading the Web: Boston Web Performance Meetup
Avatar for Jonathan Klein jklein
1
350
Upgrading the Web: Driving Support For New Standards
Avatar for Jonathan Klein jklein
1
870
Northeastern Lunch and Learn
Avatar for Jonathan Klein jklein
0
290
Profiling PHP With XHProf
Avatar for Jonathan Klein jklein
1
1.1k

Other Decks in Technology

See All in Technology
ルールやカスタム機能、どう活かす?ハンズオンで体感するIBM Bobの出力コントロール
Avatar for mu (Mizuho Uehara) muehara
1
150
SIer20年！ 培ったスキルがスタートアップで輝く時
Avatar for しゅういちろ shucho0103
0
850
200個のGitHubリポジトリを横断調査したかった
Avatar for Issei.Komori icck
0
120
Amazon Bedrock AgentCore ワークショップ JAWS UG TOHOKU / amazon-bedrock-agentcore-workshop-jawsug-tohoku-2026
Avatar for geeawa gawa
9
780
FinOps × AIエージェントで実現する コストインシデントの自動調査
Avatar for T.REX oasis1994liveforever
0
130
気づかぬうちにセキュリティ負債を生むAPIキー運用
Avatar for Michitaka Sugawara sgwrmctk
0
120
チームで進めるAI駆動アジャイル×ウォーターフォール
Avatar for 熊井悠 kumaiu
0
160
エンジニアリング戦略の作り方 / Crafting Engineering Strategy
Avatar for iwashi iwashi86
21
6.8k
How Timee Delivers Day 1 Production Ready LLM Features
Avatar for tomoyks tomoyks
0
190
【NRUG vol.18】KubernetesにおけるNew Relicデータ取得量削減の考え方
Avatar for NRUG member nrug_member
0
110
LLMにもCAP定理があるという話
Avatar for Haruka Sakihara harukasakihara
0
320
AWSシリコン最前線 〜AI時代のチップ選択を読み解く〜
Avatar for Hiroshi Tokoyo htokoyo
2
550

Featured

See All Featured
The Curious Case for Waylosing
Avatar for Cassini Nazir cassininazir
1
380
Chasing Engaging Ingredients in Design
Avatar for Sebastian Deterding codingconduct
0
220
Optimising Largest Contentful Paint
Avatar for Harry Roberts csswizardry
37
3.7k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
Avatar for Vitaly Friedman smashingmag
254
22k
The agentic SEO stack - context over prompts
Avatar for schlessera schlessera
0
820
Why You Should Never Use an ORM
Avatar for John Nunemaker jnunemaker
PRO
61
9.9k
Conquering PDFs: document understanding beyond plain text
Avatar for Ines Montani inesmontani
PRO
4
2.8k
Improving Core Web Vitals using Speculation Rules API
Avatar for Sergey Chernyshev sergeychernyshev
21
1.5k
Automating Front-end Workflow
Avatar for Addy Osmani addyosmani
1370
210k
The Anti-SEO Checklist Checklist. Pubcon Cyber Week
Avatar for Ryan Jones ryanjones
0
160
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
Avatar for Aki / @LoveIdahoBurger aki_iinuma
128
56k
How GitHub (no longer) Works
Avatar for Zach Holman holman
316
150k

Transcript

  1. Scaling PHP to 40 Million Uniques Northeast PHP 2013 Jonathan

    Klein @jonathanklein Tuesday, September 24, 13

  2. Scaling PHP to 40 Million Uniques Northeast PHP 2013 Jonathan

    Klein @jonathanklein Tuesday, September 24, 13

  3. Scaling PHP to 60 Million Uniques Northeast PHP 2013 Jonathan

    Klein @jonathanklein Tuesday, September 24, 13

  4. Who Am I? Tuesday, September 24, 13

  5. Who Am I? • Performance Engineer at Etsy Tuesday, September

    24, 13

  6. Who Am I? • Performance Engineer at Etsy • Boston

    Web Perf Meetup Organizer Tuesday, September 24, 13

  7. Who Am I? • Performance Engineer at Etsy • Boston

    Web Perf Meetup Organizer • Previously at Wayfair Tuesday, September 24, 13

  8. Who Am I? • Performance Engineer at Etsy • Boston

    Web Perf Meetup Organizer • Previously at Wayfair • Led team that converted to PHP Tuesday, September 24, 13

  9. Slides, Links: jkle.in/nephp Tuesday, September 24, 13

  10. codeascraft.com Tuesday, September 24, 13

  11. Some Etsy Stats Tuesday, September 24, 13

  12. Some Etsy Stats • 1.5 billion page views/month Tuesday, September

    24, 13

  13. Some Etsy Stats • 1.5 billion page views/month • Almost

    $1B in sales last year Tuesday, September 24, 13

  14. Some Etsy Stats • 1.5 billion page views/month • Almost

    $1B in sales last year • Over 1M lines of PHP Tuesday, September 24, 13

  15. Some Etsy Stats • 1.5 billion page views/month • Almost

    $1B in sales last year • Over 1M lines of PHP • 60M+ unique visitors/month Tuesday, September 24, 13

  16. 2012 Traffic Tuesday, September 24, 13

  17. August 2012 Traffic Tuesday, September 24, 13

  18. August 2012 Traffic 50% Higher Tuesday, September 24, 13

  19. Holidays: ~90M Uniques Tuesday, September 24, 13

  20. Tuesday, September 24, 13

  21. Tuesday, September 24, 13

  22. A few others... Tuesday, September 24, 13

  23. A few others... • Search (Solr) Tuesday, September 24, 13

  24. A few others... • Search (Solr) • Gearman Tuesday, September

    24, 13

  25. A few others... • Search (Solr) • Gearman • Redis

    Tuesday, September 24, 13

  26. A few others... • Search (Solr) • Gearman • Redis

    • Postgres (legacy) Tuesday, September 24, 13

  27. Tuesday, September 24, 13

  28. Hardware (Supermicro) Tuesday, September 24, 13

  29. Hardware (Supermicro) 2x 8-core Intel E5-2960 CPUs Tuesday, September 24,

    13

  30. Hardware (Supermicro) 2x 8-core Intel E5-2960 CPUs 24GB of RAM

    Tuesday, September 24, 13

  31. Hardware (Supermicro) 2x 8-core Intel E5-2960 CPUs 24GB of RAM

    160GB SSD Tuesday, September 24, 13

  32. Apache Tuesday, September 24, 13

  33. Apache • Apache 2.2 Tuesday, September 24, 13

  34. Apache • Apache 2.2 • Prefork MPM Tuesday, September 24,

    13

  35. Apache • Apache 2.2 • Prefork MPM • mod_php5 Tuesday,

    September 24, 13

  36. Apache • Apache 2.2 • Prefork MPM • mod_php5 StartServers

    30 MinSpareServers 30 MaxSpareServers 60 ServerLimit 60 MaxClients 60 MaxRequestsPerChild 0 Tuesday, September 24, 13

  37. PHP Tuesday, September 24, 13

  38. PHP • PHP 5.4 Tuesday, September 24, 13

  39. PHP • PHP 5.4 • Zend OPCache (3GB Memory Segment)

    Tuesday, September 24, 13

  40. PHP • PHP 5.4 • Zend OPCache (3GB Memory Segment)

    • memory_limit: 128M Tuesday, September 24, 13

  41. PHP • PHP 5.4 • Zend OPCache (3GB Memory Segment)

    • memory_limit: 128M • max_execution_time: 30 Tuesday, September 24, 13

  42. Optimizing PHP http://www.slideshare.net/jnklein/northeast-php-high-performance-php Tuesday, September 24, 13

  43. Optimizing PHP • See last year’s talk http://www.slideshare.net/jnklein/northeast-php-high-performance-php Tuesday, September

    24, 13

  44. Optimizing PHP • See last year’s talk • Use an

    Opcode Cache http://www.slideshare.net/jnklein/northeast-php-high-performance-php Tuesday, September 24, 13

  45. Optimizing PHP • See last year’s talk • Use an

    Opcode Cache • xhprof http://www.slideshare.net/jnklein/northeast-php-high-performance-php Tuesday, September 24, 13

  46. Optimizing PHP • See last year’s talk • Use an

    Opcode Cache • xhprof • StatsD/Graphite http://www.slideshare.net/jnklein/northeast-php-high-performance-php Tuesday, September 24, 13

  47. Optimizing PHP • See last year’s talk • Use an

    Opcode Cache • xhprof • StatsD/Graphite • Find hotspots http://www.slideshare.net/jnklein/northeast-php-high-performance-php Tuesday, September 24, 13

  48. Optimizing PHP • See last year’s talk • Use an

    Opcode Cache • xhprof • StatsD/Graphite • Find hotspots • Upgrade it http://www.slideshare.net/jnklein/northeast-php-high-performance-php Tuesday, September 24, 13

  49. Static Arrays Tuesday, September 24, 13

  50. Translations class language_de { static $translations = array( "<hash>" =>

    array("content" => 'Accessoires', "file" => "TAXONOMY"), "<hash>" => array("content" => 'Keramik', "file" => "CATEGORY"), "<hash>" => array("content" => 'Dekorieren', "file" => "CATEGORY"), "<hash>" => array("content" => 'Getaggt %s', "file" => "Foo.php"), "<hash>" => array("content" => 'Badezusatz', "file" => "CATEGORY"), "<hash>" => array("content" => 'Datum:{% $date %}', "file" => "bar.tpl") ... snip ... ); } Tuesday, September 24, 13

  51. Tuesday, September 24, 13

  52. Tuesday, September 24, 13

  53. Atomic Deploys Tuesday, September 24, 13

  54. Deploying PHP http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/ Tuesday, September 24, 13

  55. Deploying PHP • mod_realdoc - Apache module http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/ Tuesday, September

    24, 13

  56. Deploying PHP • mod_realdoc - Apache module • incpath -

    PHP Extension http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/ Tuesday, September 24, 13

  57. Deploying PHP • mod_realdoc - Apache module • incpath -

    PHP Extension • A/B symlink swap http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/ Tuesday, September 24, 13

  58. Deploying PHP • mod_realdoc - Apache module • incpath -

    PHP Extension • A/B symlink swap • Avoid recompilation to opcodes http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/ Tuesday, September 24, 13

  59. Upgrading PHP Tuesday, September 24, 13

  60. PHP 5.4 vs. 5.3 - CPU PHP 5.3 PHP 5.4

    Tuesday, September 24, 13

  61. PHP 5.4 vs. 5.3 - Memory PHP 5.3 PHP 5.4

    Tuesday, September 24, 13

  62. Tuesday, September 24, 13

  63. Understand Framework Overhead Tuesday, September 24, 13

  64. http://systemsarchitect.net/performance-benchmark-of-popular-php-frameworks/ Tuesday, September 24, 13

  65. Tuesday, September 24, 13

  66. Tuesday, September 24, 13

  67. Tuesday, September 24, 13

  68. Frameworks are Harder to Scale Tuesday, September 24, 13

  69. Tuesday, September 24, 13

  70. Memcached Tuesday, September 24, 13

  71. Memcached • Roughly same hardware as webs Tuesday, September 24,

    13

  72. Memcached • Roughly same hardware as webs • More memory

    (48GB) Tuesday, September 24, 13

  73. Memcached • Roughly same hardware as webs • More memory

    (48GB) • Shard keys across servers Tuesday, September 24, 13

  74. Memcached • Roughly same hardware as webs • More memory

    (48GB) • Shard keys across servers • mctop - https://github.com/etsy/mctop Tuesday, September 24, 13

  75. Tuesday, September 24, 13

  76. MySQL Tuesday, September 24, 13

  77. MySQL • MySQL 5.5 (http://jkle.in/nephp) Tuesday, September 24, 13

  78. MySQL • MySQL 5.5 (http://jkle.in/nephp) • ~20 master/master pairs Tuesday,

    September 24, 13

  79. MySQL • MySQL 5.5 (http://jkle.in/nephp) • ~20 master/master pairs •

    innodb Tuesday, September 24, 13

  80. MySQL • MySQL 5.5 (http://jkle.in/nephp) • ~20 master/master pairs •

    innodb • thread_cache=800 Tuesday, September 24, 13

  81. MySQL • MySQL 5.5 (http://jkle.in/nephp) • ~20 master/master pairs •

    innodb • thread_cache=800 • max_connections=2500 Tuesday, September 24, 13

  82. MySQL • MySQL 5.5 (http://jkle.in/nephp) • ~20 master/master pairs •

    innodb • thread_cache=800 • max_connections=2500 • [Almost] no joins Tuesday, September 24, 13

  83. Theoretically we can serve ~9000 req/sec Tuesday, September 24, 13

  84. Tuesday, September 24, 13

  85. Static Content Tuesday, September 24, 13

  86. Tuesday, September 24, 13

  87. ~500 Million Images Tuesday, September 24, 13

  88. Offloading Requests Tuesday, September 24, 13

  89. Offloading Requests • Multiple CDNs Tuesday, September 24, 13

  90. Offloading Requests • Multiple CDNs • Img hit rate >

    90% Tuesday, September 24, 13

  91. Offloading Requests • Multiple CDNs • Img hit rate >

    90% • CSS/JS hit rate ~100% Tuesday, September 24, 13

  92. Offloading Requests • Multiple CDNs • Img hit rate >

    90% • CSS/JS hit rate ~100% • Edge serves ~7000 reqs/sec Tuesday, September 24, 13

  93. What about misses? Tuesday, September 24, 13

  94. Tuesday, September 24, 13

  95. Change Management Tuesday, September 24, 13

  96. Feature Flags Tuesday, September 24, 13

  97. Tuesday, September 24, 13

  98. // Owner: Username $server_config['foo']['bar'] = array( 'users' => array('jsmith', 'jdoe'),

    ); // Owner: Username $server_config['baz'] = [ 'enabled' => [ 'variant1' => 0, 'variant2' => 0, 'variant3' => 50 ], ]; // Owner: Username $server_config['qux'] = array( 'enabled' => 100, 'data' => array( 'somedata' ) ); Tuesday, September 24, 13

  99. if (Feature::isEnabled('foo.bar')) { // Code } Tuesday, September 24, 13

  100. Ramp-up: 1%, 5%, 25%, 100% Tuesday, September 24, 13

  101. Deployinator Tuesday, September 24, 13

  102. Tuesday, September 24, 13

  103. Schemanator Tuesday, September 24, 13

  104. Tuesday, September 24, 13

  105. Recent Improvements Tuesday, September 24, 13

  106. Tuesday, September 24, 13

  107. Sandy Bridge Tuesday, September 24, 13

  108. Baseline Performance (median) Tuesday, September 24, 13

  109. Small code change Baseline Performance (median) Tuesday, September 24, 13

  110. Small code change PHP 5.4 Baseline Performance (median) Tuesday, September

    24, 13

  111. Small code change PHP 5.4 Disabled Hyperthreading Baseline Performance (median)

    Tuesday, September 24, 13

  112. Challenges Tuesday, September 24, 13

  113. Tuesday, September 24, 13

  114. Options Tuesday, September 24, 13

  115. Options • Scale Horizontally (more shards) Tuesday, September 24, 13

  116. Options • Scale Horizontally (more shards) • Scale Vertically (SSD,

    bigger drives) Tuesday, September 24, 13

  117. Options • Scale Horizontally (more shards) • Scale Vertically (SSD,

    bigger drives) • Change architecture Tuesday, September 24, 13

  118. Tuesday, September 24, 13

  119. Architecture Changes Tuesday, September 24, 13

  120. Architecture Changes • Ongoing conversation Tuesday, September 24, 13

  121. Architecture Changes • Ongoing conversation • Separate logical/physical shards Tuesday,

    September 24, 13

  122. Architecture Changes • Ongoing conversation • Separate logical/physical shards •

    Complex Tuesday, September 24, 13

  123. Architecture Changes • Ongoing conversation • Separate logical/physical shards •

    Complex • Server failure Tuesday, September 24, 13

  124. Architecture Changes • Ongoing conversation • Separate logical/physical shards •

    Complex • Server failure • Migrating data Tuesday, September 24, 13

  125. Architecture Changes • Ongoing conversation • Separate logical/physical shards •

    Complex • Server failure • Migrating data • Consistency Tuesday, September 24, 13

  126. Monitoring Tuesday, September 24, 13

  127. Graphite Tuesday, September 24, 13

  128. Stacked Search Timers Tuesday, September 24, 13

  129. Ganglia Tuesday, September 24, 13

  130. Tuesday, September 24, 13

  131. Nagios Tuesday, September 24, 13

  132. Tuesday, September 24, 13

  133. Tuesday, September 24, 13

  134. Takeaways Tuesday, September 24, 13

  135. K.I.S.S. Tuesday, September 24, 13

  136. Use Proven Technologies Tuesday, September 24, 13

  137. Tuesday, September 24, 13

  138. Understand Your Stack Tuesday, September 24, 13

  139. Measure Everything Tuesday, September 24, 13

  140. Work at Etsy ;-) Tuesday, September 24, 13

  141. Connect http://www.meetup.com/Web-Performance-Boston/ www.etsy.com/careers [email protected] @jonathanklein Tuesday, September 24, 13