uSwitch.com - Our Age of Enlightenment

My talk from the NoSQL Exchange, London, 28th November 2012. Hosted by Skillsmatter.

Variation and selection are great ways to evolve complex systems. Composing complex systems from smaller pieces allows the introduction of greater variation. Maximise discovery.

Homogeneity in the large should be avoided. Models provide utility- both organisationally and technically. Rather than trying to view the world as a single model under tension, compose many smaller models for greater utility.

NoSQL represents the rejection of single models. Composing a system with multiple data stores allows us to safely experiment and have data stores 'find their place'.

Transcript

1. Paul Ingles @pingles paul@forward.co.uk uSwitch.com Our Age of Enlightenment
3. ... a cultural movement of intellectuals in 18th century ... [whose] purpose was to reform society using reason (rather than tradition, faith and revelation) The Age of Enlightenment
4. Enlightenment at
5. A/B Testing
6. Operational Transparency
7. Tool Hacking
8. Cross-sectional Study
9. Machine Learning
10. Enlightenment at
11. application of reason over intuition, tradition and faith Enlightenment at ... inhibited by legacy monolithic system ... reinforced through inflexibile models
12. NoSQL (and its philosophy) Micro-service Architecture This talk Enlightenment Enablement
13. inhibits innovation restricts variation Monolithic Architecture
14. Model is Utility
15. Grand Unified Model Has to be general Limits utility Continually under tension
16. Single Model in Tension: Energy
17. Single Model in Tension: Energy
18. Single Model in Tension: Energy
19. Single Model in Tension: Energy
20. Single Model in Tension: DB
21. MS SQL Server Operational Store Reporting Data Warehousing Network Analysis Correlational Study Realtime Single Model in Tension: DB
22. MS SQL Server Change is global Releasing is complex Add process to manage complexity Single Model in Tension: DB
23. hard to avoid if difficult to talk to anything else Monolithic Architecture make decisions on tradition and faith
24. NoSQL Micro-service Architecture Variation
25. “the mechanics of genetic evolution provide a clear and compelling picture of how a system can capture and exploit implicit insight without creating explicit knowledge... ... trial and error advances society’s material interests without conscious knowledge or planning.” - Jim Manzi, Uncontrolled
26. NoSQL No Variation in the system Reject artificial constraint Right tool for the job
27. Time Knowledge
28. NoSQL is obvious “The solutions are all simple - after you have arrived at them. But they’re only simple when you know already what they are.” -Zen and the Art of Motorcycle Maintenance
29. Micro-service Architecture
31. Keep problems local Build complex system from smaller parts Small flows of data between independent processes
32. Unix Systems Transport Pipes HTTP Representation Text Data Process Composition
33. Micro-service and NoSQL: Energy
34. Tariff Editor Comparison Service Energy Website Back Office
35. Tariff Editor Comparison Service Energy Website Back Office MongoDB MongoDB Charon SQL Server MySQL Energy Region CSV Reconciliation Profit & Loss Southampton Uni Research MysQL Redis MySQL:RDS
36. Tariff Editor Comparison Service Energy Website Back Office Car Insurance Website Credit Cards Website ... nginx proxy servers Kaa HDFS Hive Cascalog Visitor Service Neo4j Cassandra
37. Micro-service Architecture Platform for continuous delivery/ strangulation Bounded context guides system decomposition Keep problems local Composition with narrow APIs and data
38. NoSQL Data stores become implementation detail Locality makes decisions trivial “Data stores find their place”
39. NoSQL Micro-service Architecture Variation and Selection Composition
40. Enlightenment at
41. Enlightenment at NoSQL: Rejection of “One True” Model System composition important Embrace variation and selection: Monoliths and homogeneity considered harmful. Micro-services make it safe. Compose with data flows and narrow APIs
42. Related Material http://www.slideshare.net/jamesalewis/java-microservices http://oobaloo.co.uk/ http://vimeo.com/45136211 http://skillsmatter.com/podcast/scala/reviving- uswitchs-back-office-with-clojure
43. Thank you @pingles paul@forward.co.uk http://github.com/pingles
44. Questions :) @pingles paul@forward.co.uk http://github.com/pingles