Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scaling Webpack to Thousands of Concurrent Builds

Scaling Webpack to Thousands of Concurrent Builds

Building JavaScript for the web is easier than it's ever been with babel, webpack & rollup doing all the heavy lifting for individual builds – but what if you need a lot of builds across multiple environments like development, test, and production? Introduced at Node.js Interactive 2016 Warehouse.ai is an open-source distributed build system supporting webpack, babel, and browserify that aims to make Serverless front-end deployments & rollback easy.

This talk will cover the challenges (and solutions) for Warehouse to scale webpack & npm install to thousands of daily builds with high bursts of concurrency during peak hours.

Charlie Robbins

October 11, 2018
Tweet

More Decks by Charlie Robbins

Other Decks in Technology

Transcript

  1. SENIOR DIR., UX PLATFORM @ GoDaddy BOARD MEMBER, NODE.JS FOUNDATION

    FORMERLY FOUNDER at NODEJITSU @INDEXZERO GITHUB TWITTER SO WHAT’S ALL THIS THEN?
  2. What am I going to talk about? 5 1 Define

    the problem 2 Webpack it up, webpack it in 3 Initial solutions & naive approaches 4 DNS & HTTP Starvation 5 Putting it all together
  3. from a pool of servers Everyone’s computer npm install requests

    webpack requests DNS Caching overloads one machine