Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Managing Machine Learning Models in Production ...

Managing Machine Learning Models in Production - Strata Singapore 2017

Slides of talk "Managing Machine Learning Models in Production" presented at Strata Data Conference in Singapore.

Anand Chitipothu

December 06, 2017
Tweet

More Decks by Anand Chitipothu

Other Decks in Technology

Transcript

  1. Who is Speaking? Anand Chitipothu @anandology Co-founder and platform architect

    of @rorodata Worked at Internet Archive & Strand Life Sciences Advanced programming courses at @pipalacademy 2
  2. Machine Learning [Machine learning is the] field of study that

    gives computers the ability to learn without being explicitly programmed. — Arthur Samuel 3
  3. Challenges: Build — Compute: switch between different compute needs seamlessly

    — Setup: installing require software dependencies — Automation: re-train models periodically 9
  4. Challenges: Deploy — Designing: Designing and documenting APIs — Wiring:

    Running the service and configuring URL endpoints — Scaling: Scale to meet the usage 10
  5. Challenges: Manage — Model versions: Keep track of multiple versions

    of models — Model metadata: What went into building each model? — Collaboration: Sharing models in and across the teams 11
  6. Challenges: Monitor — Performance: Request rate & Latency — Logging:

    Record requested data and predicted values — Effectiveness: Model accuracy in production 12
  7. The pace of innovation of a data-driven business is limited

    by the bottlenecks in their data science workflows 13
  8. How to Solve it? — Duct tape approach — Build

    a data platform — Using a managed platform-as-a-service 14
  9. The rorodata Platform Cloud platform to address the challenges in

    managing machine learning in production. 15
  10. rorodata: Build — Elastic Compute $ roro run -size M2

    python train.py Created new job b42c12a0 $ roro run:notebook --gpu Jupyter notebook is available at: https://a098cd79.rorocloud.io/ 16
  11. rorodata: Build — Elastic Compute — Seamless Setup # roro.yml

    runtime: python3-keras # requirements.txt hyperopt>=0.1 17
  12. rorodata: Build — Elastic Compute — Seamless Setup — Flexible

    Scheduling # roro.yml runtime: python3-keras tasks: - name: retrain command: python train.py when: "every day at 10:00 AM" 18
  13. rorodata: Deploy — Python functions as API # roro.yml services:

    - name: api function: predict.predict - name: rating function: credit_grade.get_credit_grade 19
  14. rorodata: Deploy — Python functions as API — Automatic endpoint

    management $ roro deploy Deploying credit-risk... Building docker image... done. Updating scheduled jobs... done. Restarting services... api: https://credit-risk--api.rorocloud.io/ rating: https://credit-risk--rating.rorocloud.io/ Deployed v4 of credit-risk project. 20
  15. rorodata: Deploy — Python functions as API — Automatic endpoint

    management — Simple Scaling # roro.yml services: - name: api function: predict.predict size: M2 21
  16. rorodata: Manage — Model Metadata Model-ID: c54e00eb Model-Name: iris Model-Version:

    3 Date: 2017-08-02T10:20:30Z ... Input-Source: s3://iris-sample-data Dataset-Features: Sepal-Length,Sepal-Width,Petal-... Training-Algorithm: SVM Training-Parameters: C=10; alpha=0.4; kernel=rbf Training-Accuracy: 0.85 22
  17. rorodata: Manage — Model Metadata — Model Versioning +---------------------------------------+ |

    Model Repository A | | | | ModelImage - v1 ModelImage - v2 | | +---------------+ +---------------+ | | | Model v1 | | Model v2 | | | +---------------+ +---------------+ | | | Metadata v1 | | Metadata v2 | | | +---------------+ +---------------+ | +---------------------------------------+ 23
  18. rorodata: Manage — Model Metadata — Model Versioning - Python

    API project = roro.get_current_project() repo = project.get_model_repository("iris") model_image = repo.new_image(model) model_image["Dataset-Features"] = "Sepal-Length,Sepal-Width,Petal-..." model_image["Training-Accuracy"] = 0.85 model_image.save(comment="Built a new model using SVM") 24
  19. rorodata: Manage — Model Metadata — Model Versioning — Collaboration

    Add other people as collaborators to your project. 25
  20. rorodata: Monitor — Performance Metrics — Data Logging # Data

    Logger to record key-value pairs. roro.record("credit-risk-api", model_name="credit-risk", request_id=request_id, user_id=user_id, input=input_data, predicted=predicted_value) 27
  21. rorodata: Monitor — Performance Metrics — Data Logging — Feedback

    Loop API endpoint to record the user action to close the feedback loop. This allows computing the effectiveness of the live model. 28
  22. Solving Machine Learning in Production — Start early — Take

    a platform approach — Learn from others 30
  23. Our Open Source Components Firefly Function as service. https://github.com/rorodata/firefly rorolite

    Lite version of the platform to deploy ML apps on your server. https://github.com/rorodata/rorolite 31