Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Smart, Scalable Content Distribution @OReilly S...

Smart, Scalable Content Distribution @OReilly Software Architecture New York

Separating data from functionality is one of the tricks to speedier applications, since it allows accessing the right content at the right time. Partitioning data cleverly presents several challenges that I have overcome on several different projects, and it requires a combination of architecture, DevOps, development, and testing skills that is pretty unique. Unless you have such a Swiss Army knife on your team (and are not afraid of the bus factor), you’ll want to come listen to how I do it.

I share a solution to content snapshotting, distribution, and caching in a silo-based architecture involving tens of machines, walking you through the problems encountered and solutions I came up with. You’ll leave knowing how to automate content distribution on individual nodes and warm up caches, all while keeping your website permanently online.

Georgiana Gligor

April 04, 2017
Tweet

More Decks by Georgiana Gligor

Other Decks in Technology

Transcript

  1. 6 Georgiana Gligor ✤ Geek. Mother. Do-er. ✤ Crafting enterprise

    apps ✤ Large-Scale Application Architect ✤ PhD Student in Systems Engineering @gbtekkie [email protected]
  2. 11 CAPTURING CONTENT SECURITY pen & paper structured data user

    interface non-structured digital documents MANUAL VERSIONING
  3. 14 TYPICAL MODERN NEEDS collaboration UI transactional UI YOUR CONTROL

    internal UI APIs digital files content pen & paper internal UI media UI SaaS services APIs 3RD
 PARTY content
  4. CONTENT SOURCES 23 3rd party API content internal API internal

    API CURRENT STATE load balancer frontend middleware middleware middleware cache frontend
  5. CONTENT SOURCES 25 traffic controller silo 1 silo 3 silo

    2 3rd party API content internal API internal API MEET AND GREET
  6. 26 traffic controller variation A variation C variation B THE

    CATCH CONTENT SOURCES 3rd party API content internal API internal API
  7. 27 traffic controller AWS Google private cloud MULTI - CLOUD

    CONTENT SOURCES 3rd party API content internal API internal API
  8. 28 Phrase Of The Day Each day is an opportunity

    It's one thing to say this phrase each day, another to actually apply it. When you're standing in line at Starbucks, remember that this day has never existed before.
  9. 32 traffic controller v5.1.9 v11.4.7 v5.2.0 v11.4.7 v6.0.1 v12.0.1 CONTENT

    VERSIONING CONTENT SOURCES 3rd party API transactional internal API
  10. 35 content CMS UI APIs import import 3RD PARTY DATA

    IMPORT 3rd party DIFFERENT STRUCTURE
  11. 37 content import APIs mix & match import mix &

    match 3RD PARTY CONTENT 3rd party
  12. 38 content import APIs mix & match import mix &

    match CMS UI manually resolve data conflicts 3RD PARTY CONTENT 3rd party