Architectural Patterns in Machine Learning to Generate Sustainable Business Value

In this talk, I aim to highlight common concerns with building and deploying Machine Learning solutions, and discuss various patterns that can be consistently used to deliver value sustainably and quickly across a broad range of industries and specific business problems.
We will dive into some common patterns such as building composable components for reusability and consistency, production readiness, optimizing for learning with deployed solutions and guidelines for investments in research, as well as discuss tradeoffs against different dimensions such as latency vs complexity, generalizability of modeling solutions vs problem specific optimization. You should walk away from this talk with some general heuristics and patterns that make decisions for your ML solutions simpler, easier to manage and more successful.

Transcript

1. Architectural Patterns in Machine Learning Nishan Subedi VP, Algorithms @ Overstock, Inc.
2. Overstock.com
3. What is a Pattern? Relationships between components which resolves a small system of interacting and conflicting forces, and is independent of all other forces in all possible contexts. - Components can be created independently. - Components can then be combined together. - Design is an error correcting process. - Good design is characterized by absence of misfits (painful to do something). Many subsets of forces work together to build a pattern.
6. Driven by the Scientific Methodology (OODA Loop)
7. Scientific Methodology (OODA Loop) applied to the ML Product life cycle
8. Decide Every action starts with a decision
9. Experimentation requires flexibility to test new • Model architectures • Featurization & data aggregations • Optimization functions • Training data sampling • Evaluation metrics
10. Changing environment as we Act on what we Decide
11. Abstraction allows preserving context across changing environments https://www.tecton.ai/blog/what-is-a-feature-store/
12. Production Forces: • Latency • Availability • Consistency • Operability
13. Consistent model deployment abstraction https://www.tensorflow.org/tfx/serving/architecture
14. Observe
15. Forces at play during observation • A/B testing • Data drift • Hidden feedback loops • Feature erosion • Consistency of features across batch / real time contexts • Bias • Offline / online metric comparison
16. Forms that facilitate Observation - Variant
17. Designing for Observability
18. Value = timely response to changing needs aka faster iterations through the OODA loop “Whoever can handle the quickest rate of change is the one who survives.” ― John Boyd
19. Architect for ease of iteration on the OODA loop Encode good practices into the architecture, make observability a default.
20. Reusability
21. Pattern Language Each pattern forms a word or thought of a language rather than a prescriptive way to design or solve a problem. This language allows precise conversations in teams, organizations and the industry. Architecture allows you to transcend a specific context to communicate with others within similar forces.
22. Organizational patterns No pattern is an isolated entity. Each pattern can exist in the world, only to the extent that it is supported by other patterns: the larger patterns in which it is embedded. You and what your architecture is part of your organization.
23. Organizational patterns An organization is made of the conversations between its people - actions emerge from these. An organization’s ability to create language is synonymous with its ability to evolve.
24. Organizational patterns Having an architecture gives you a language allows you to get faster iterations of your OODA loop since it allows to much more effectively shine light on the right aspects that you need to talk about.
25. How to speed up your OODA loop? 1. Consistent architecture 2. Encapsulation of higher order information through data refinery 3. Flexibility in optimization well integrated with productionization process 4. Variant identification and propagation through various stages 5. Codify life cycle of ML in the language & process of the company
26. Thankyou! We are hiring: overstock.com/careers