Smart, Scalable Content Distribution @OReilly Software Architecture New York

Separating data from functionality is one of the tricks to speedier applications, since it allows accessing the right content at the right time. Partitioning data cleverly presents several challenges that I have overcome on several different projects, and it requires a combination of architecture, DevOps, development, and testing skills that is pretty unique. Unless you have such a Swiss Army knife on your team (and are not afraid of the bus factor), you’ll want to come listen to how I do it.

I share a solution to content snapshotting, distribution, and caching in a silo-based architecture involving tens of machines, walking you through the problems encountered and solutions I came up with. You’ll leave knowing how to automate content distribution on individual nodes and warm up caches, all while keeping your website permanently online.

Transcript

1. SMART, SCALABLE CONTENT DISTRIBUTION GEORGIANA GLIGOR Tekkie Consulting @gbtekkie
2. 2 ✤ CMS comparison criteria ✤ CMS team roles ✤ content modeling
3. 3 ✤ CMS comparison criteria ✤ CMS team roles ✤ content modeling Not Today
4. 4 NOT ADDRESSING TODAY content management systems content delivery networks build a one-size-fits-all recipe
5. 5 WE WILL AIM FOR automate content distribution content partitioning high availability
6. 6 Georgiana Gligor ✤ Geek. Mother. Do-er. ✤ Crafting enterprise apps ✤ Large-Scale Application Architect ✤ PhD Student in Systems Engineering @gbtekkie gb@tekkie.ro
7. 7 awareness interest decision action MARKETING FUNNEL
8. 8 AGENDA COMPLEXITY DISTRIBUTION SILOS
9. CONTENT COMPLEXITY
10. 10 What business are you in? ✤ Collaboration ✤ Transactional ✤ Media property ✤ SaaS application
11. 11 CAPTURING CONTENT SECURITY pen & paper structured data user interface non-structured digital documents MANUAL VERSIONING
12. 12 EXPOSING CONTENT SCALING CONSISTENCY structured structured APIs non-structured user interface public-facing UI
13. 13 TRANSACTIONAL APPLICATION transactional UI PCI/PII COMPLIANCE structured structured APIs non-structured internal UI
14. 14 TYPICAL MODERN NEEDS collaboration UI transactional UI YOUR CONTROL internal UI APIs digital files content pen & paper internal UI media UI SaaS services APIs 3RD
 PARTY content
15. 15 TYPICAL MODERN NEEDS collaboration UI transactional UI digital files content pen & paper media UI
16. 16 PUT CONTENT FIRST transactional UI collaboration UI digital files content pen & paper media UI
17. 17 TRANSACTIONAL CONTENT pricing technicalities availability media VERY RARELY RARELY INFREQUENTLY FREQUENTLY
18. 18 COLLABORATION CONTENT Q & A reviews customer content DEMAND SPIKES DEMAND SPIKES DEMAND SPIKES
19. 19 Software industry is built around anticipating change.
20. SILOS
21. 21 Departments inside a company don’t work together. - Silo Mentality Not Today
22. 22 IMPLEMENTING CHANGE
23. CONTENT SOURCES 23 3rd party API content internal API internal API CURRENT STATE load balancer frontend middleware middleware middleware cache frontend
24. CONTENT SOURCES 24 end-to-end functionality 3rd party API content internal API internal API CURRENT STATE
25. CONTENT SOURCES 25 traffic controller silo 1 silo 3 silo 2 3rd party API content internal API internal API MEET AND GREET
26. 26 traffic controller variation A variation C variation B THE CATCH CONTENT SOURCES 3rd party API content internal API internal API
27. 27 traffic controller AWS Google private cloud MULTI - CLOUD CONTENT SOURCES 3rd party API content internal API internal API
28. 28 Phrase Of The Day Each day is an opportunity It's one thing to say this phrase each day, another to actually apply it. When you're standing in line at Starbucks, remember that this day has never existed before.
29. 29 traffic controller code content SILO COMPOSITION content sources
30. 30 traffic controller SILO’D CONTENT content sources
31. 31 traffic controller v11.4.7 v12.0.1 v12.0.2 CODE VERSIONING content sources
32. 32 traffic controller v5.1.9 v11.4.7 v5.2.0 v11.4.7 v6.0.1 v12.0.1 CONTENT VERSIONING CONTENT SOURCES 3rd party API transactional internal API
33. CONTENT DISTRIBUTION
34. 34 CMS UI CMS TO CAPTURE CONTENT content
35. 35 content CMS UI APIs import import 3RD PARTY DATA IMPORT 3rd party DIFFERENT STRUCTURE
36. 36 CONTENT NORMALIZATION
37. 37 content import APIs mix & match import mix & match 3RD PARTY CONTENT 3rd party
38. 38 content import APIs mix & match import mix & match CMS UI manually resolve data conflicts 3RD PARTY CONTENT 3rd party
39. 39 content PARTITIONING rarely frequently
40. 40 content rarely frequently CONTENT AS CODE
41. 41 content CONTENT AS CODE rarely frequently
42. AUTOMATED DISTRIBUTION 42 content traffic controller CMS UI
43. ONE MORE THING…
44. 44 2 3 EVENTUAL CONSISTENCY
45. CONTENT SOURCES 45 traffic controller CONSISTENCY MODELS transactional non-transactional
46. 46 Men have become the tools of their tools. - Henry David Thoreau
47. ‹#› THANK YOU GEORGIANA GLIGOR } Affordable quality system architecture. In-house polyglot trainings. gb@tekkie.ro
48. 48 traffic controller TOO FREQUENTLY UPDATED content
49. 49 traffic controller TOO FREQUENTLY UPDATED message queues content