Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Making Maps: The Role of Frontend Infrastructur...

Making Maps: The Role of Frontend Infrastructure at Etsy

Video: http://vimeo.com/109912254
Given October 9th at Fronteers 2014 in Amsterdam.

The scope of Etsy's frontend is massive - our JavaScript codebase has grown by 50% in the last year to more than 3,000 files totaling almost 800,000 lines. Because of Etsy's dedication to continuous deployment, the code running in production changes 25 to 50 times every day. Because of our experimentation-driven development cycle, there may be multiple production versions of our features at any given time. These factors can lead to uncertainty and fear with rolling out upgrades, deleting old code, or confidently making changes. I share how the Frontend Infrastructure team works within this ecosystem to mitigate those risks, manage the asset build pipeline, build tools to understand our frontend, automate migrations & delete as much code as possible.

Daniel Espeset

October 09, 2014
Tweet

More Decks by Daniel Espeset

Other Decks in Programming

Transcript

  1. 4

  2. pages/search_results.php if (Feature::isEnabled('my_new_feature')) { // TODO build this part }

    else { // Existing code } 20 21 22 .. 100 12 config/production.php // Owner: despeset, frontend infrastructure server_config['my_new_feature'] = ( 'enabled' => 0 ); 890 891 892 893
  3. 13 desp .join pushbot has changed topic to: "<prod> seth

    dna* gio* | desp" pushbot desp jklein : You're up desp .in pushbot has changed topic to: "desp jklein" pushbot has changed topic to: "desp* jklein" jklein .in pushbot has changed topic to: "desp* jklein*" pushbot desp jklein : everyone is ready
  4. 21 config/production.php // Owner: despeset, frontend infrastructure server_config['my_new_feature'] = (

    'enabled' => 0, 'group' => 9903, 'admin' => 'on' ); 890 891 892 893 894 895
  5. 22 config/production.php // Owner: despeset, frontend infrastructure server_config['my_new_feature'] = (

    'enabled' => 50, 'group' => 9903, 'admin' => 'on' ); 890 891 892 893 894 895
  6. Daniel Espeset @danielespeset 24 idea code release idea code release?

    A/B test idea prototype release A/B test validate A/B test refine 2005 2010 2012 Shamelessly stolen from Dan McKinley mcfunley.com/data-driven-products-now Evolution of Continuous Experimentation
  7. Daniel Espeset @danielespeset 25 • Engineers are trusted and have

    massive access • The production codebase is constantly changing • That codebase is filled with experiments TL;DR
  8. Daniel Espeset @danielespeset 28 At any given time we may

    be running dozens of experiments per page
  9. Daniel Espeset @danielespeset • • Wraps RequireJS AMD compiler •

    Parallelizes work across CPU cores • Serves fully compiled assets on development VMS 39 Builda [Build Assets]
  10. Daniel Espeset @danielespeset 40 Techniques for serving built assets in

    dev • Build on the fly in response to HTTP requests for asset files
  11. Daniel Espeset @danielespeset 41 Techniques for serving built assets in

    dev • Build on the fly in response to HTTP requests for asset files • Watch FS for any changes and rebuild everything
  12. Daniel Espeset @danielespeset 42 Techniques for serving built assets in

    dev • Build on the fly in response to HTTP requests for asset files • Watch FS for any changes and rebuild everything • Watch FS for any changes and only rebuild affected targets
  13. Daniel Espeset @danielespeset 43 Techniques for serving built assets in

    dev • Build on the fly in response to HTTP requests for asset files • Watch FS for any changes and rebuild everything • Watch FS for any changes and only rebuild affected targets this is... complicated
  14. 48 dependency_graph.json "mobile/modules/rwd": { "info": { "version": "20140815195505" }, "children":

    { "common/etsy.namespace": "amd" }, "parents": { "mobile/base-container": "amd" } } 201 202 203 204 205 206 207 208 209 210 211
  15. Daniel Espeset @danielespeset 50 + + JS graph render graph

    CSS graph Ranger combines 3 dependency graphs
  16. 51

  17. 52

  18. 53

  19. 54

  20. 55

  21. 56

  22. 57

  23. 58

  24. Daniel Espeset @danielespeset • Pick a stylesheet loaded on the

    page 61 Shrinkray - clientside CSS analysis
  25. Daniel Espeset @danielespeset • Pick a stylesheet loaded on the

    page • Randomly pick 50 selectors via CSSOM 62 Shrinkray - clientside CSS analysis
  26. Daniel Espeset @danielespeset • Pick a stylesheet loaded on the

    page • Randomly pick 50 selectors via CSSOM • Search for matches with document.querySelector 63 Shrinkray - clientside CSS analysis
  27. Daniel Espeset @danielespeset • Pick a stylesheet loaded on the

    page • Randomly pick 50 selectors via CSSOM • Search for matches with document.querySelector • Send results to server 64 Shrinkray - clientside CSS analysis
  28. Daniel Espeset @danielespeset • Pick a stylesheet loaded on the

    page • Randomly pick 50 selectors via CSSOM • Search for matches with document.querySelector • Send results to server • Aggregate data with Map Reduce 65 Shrinkray - clientside CSS analysis
  29. 66

  30. Daniel Espeset @danielespeset • No @extends ever • No @mixins

    (for now) • Nesting level < 4 • Lints to enforce these and strict code-style adherence 71 Restricting SCSS functionality
  31. Daniel Espeset @danielespeset • @import statements needed changed • Converter

    fixes 171,244 scss-lint errors • Had to resolve hundreds of invalid CSS rules & selectors 72 Converter script
  32. Daniel Espeset @danielespeset 76 CSS Source Legacy CSS Build Legacy

    Builda Converter SCSS Builda New CSS Build
  33. Daniel Espeset @danielespeset 77 CSS Source Legacy CSS Build Legacy

    Builda Converter SCSS Builda New CSS Build AST Differ Parse to AST Parse to AST
  34. Daniel Espeset @danielespeset 78 CSS Source Legacy CSS Build Legacy

    Builda Converter SCSS Builda 100% New CSS Build
  35. Daniel Espeset @danielespeset 79 CSS Source Legacy CSS Build Legacy

    Builda Converter SCSS Builda 100% Etsy New CSS Build
  36. Daniel Espeset @danielespeset 80 CSS Source Legacy CSS Build Legacy

    Builda Converter SCSS Builda 50% 50% New CSS Build
  37. Daniel Espeset @danielespeset 81 CSS Source Legacy CSS Build New

    CSS Build Legacy Builda Converter SCSS Builda 100%
  38. Daniel Espeset @danielespeset 87 The browser is part of your

    distributed system, not just a client you support.