Deploying Machine Learning Models to Production: Challenges & Solutions using MLOps

For the most part Machine Learning is similar to traditional Software Development and most of the principles and practices that apply to Traditional Software Development also apply to Machine Learning. But there are also certain unique challenges that come with deploying ML models to Production.

In this presentation, we will look at the top Challenges you face deploying Machine Learning Models to Production and how to tackle those Challenges using MLOps.

Key Takeaways:

* How is Machine Learning different than Traditional Software Development
* Top challenges deploying ML Models to Production
* What is MLOps and how to tackle ML specific challenges using that
* Anecdotes about deploying ML Models using industry principles and best practices

Transcript

1. Deploying Machine Learning Systems to Production Challenges & Solutions using MLOps Adarsh Shah Engineering Leader, Coach, Hands-on Architect Independent Consultant @shahadarsh shahadarsh.com
2. shahadarsh.com @shahadarsh Traditional Software Development vs Machine Learning
3. shahadarsh.com @shahadarsh Machine Learning Workflow Data Acquisition Data Preparation Model Development Model Training Model Serving Accuracy Evaluation Code
 Changes Retraining Data Management Experimentation Production Deployment
4. shahadarsh.com @shahadarsh Fun Fact about Model training Everybody has. They just don’t realize it. Have you ever helped train a ML model?
5. shahadarsh.com @shahadarsh Hidden Technical Debt in ML Systems From the paper Hidden Technical Debt in Machine Learning Systems
6. shahadarsh.com @shahadarsh Challenges Unique to ML
7. shahadarsh.com @shahadarsh #1: Data Management Data Location Large Datasets Security Compliance
8. shahadarsh.com @shahadarsh #2: Constant Research and Experimentation Code Quality Experimentation Notebooks Tracking experiments
9. shahadarsh.com @shahadarsh #3: Training Process Retraining Training Time Reproducibility
10. shahadarsh.com @shahadarsh #4: Infrastructure Requirements Edge devices GPU & 
 high density cores Costs Elasticity
11. shahadarsh.com @shahadarsh #5: Testing Model Accuracy Data Validation Model
 Bias & Fairness
12. shahadarsh.com @shahadarsh #6: Dependency Hell Dependency Hell ARM architecture
13. shahadarsh.com @shahadarsh MLOps
14. shahadarsh.com @shahadarsh MLOps MLOps = Machine Learning + DevOps People Process + Technology +
15. shahadarsh.com @shahadarsh Roles ML Researcher ML Engineer Data Engineer MLOps Engineer
16. shahadarsh.com @shahadarsh Team Structure considerations Cross functional Team Separate Data Science Team ML Platform Engineering Team
17. shahadarsh.com @shahadarsh Data pipeline Data Source A Data Acquisition A Data Preparation A Training 
 Dataset Data Validation A Data Source B Data Source N Data Acquisition B Data Acquisition N Data Preparation B Data Preparation N Data Validation B Data Validation N Input Training Process Input
18. shahadarsh.com @shahadarsh Training Pipeline Training
 Code Continuous 
 Integration Training Data Data Pipeline Pre-trained
 Weights Validation Artifact Repository Push image Training Environment Cloud, On-Prem 
 or Edge location Infra provisioning automation GPU support Monitoring/ Logging/Alerting UI or 
 command Schedule Training Bias & Fairness Testing Build & Version
 Model
19. shahadarsh.com @shahadarsh Deployment Pipeline GitOps Monitoring/ Logging/Alerting Artifact Repository Pull Model
 Image Model 
 Training Retrain Model
 (if accuracy 
 below acceptable %) Push to Master Infra provisioning automation GPU support Model Serving 
 Environment Cloud, On-Prem 
 or Edge location Deploy Model Evaluate Model Accuracy Periodic
20. shahadarsh.com @shahadarsh Platforms available
21. shahadarsh.com @shahadarsh Platforms
22. shahadarsh.com @shahadarsh Kubeflow
23. shahadarsh.com @shahadarsh To sum it up • Machine Learning Workflow • Traditional Software Development vs Machine Learning • Unique ML Challenges • Data Management • Constant Research and Experimentation • Training Process • Infrastructure Requirements • Testing • Dependency Hell • MLOps • Roles & Team Structure Considerations • Data, Training & Deployment Pipeline • Platforms Available
24. Questions Adarsh Shah Engineering Leader, Coach, Hands-on Architect Independent Consultant @shahadarsh shahadarsh.com