Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Bridging the gap in Machine Learning on Kubernetes

Bridging the gap in Machine Learning on Kubernetes

Michael Hausenblas

April 19, 2018
Tweet

More Decks by Michael Hausenblas

Other Decks in Technology

Transcript

  1. Bridging the gap in Machine Learning on Kubernetes Michael Hausenblas

    @mhausenblas
 Developer Advocate, Red Hat
 2018-04-19, Microservices Zurich
  2. Hit me up on Twitter: @mhausenblas 2 • Developer Advocate

    @ Red Hat (Go, Kubernetes, OpenShift) • Developer Advocate @ Mesosphere (Mesos, Kubernetes) • Chief Data Engineer @ MapR (Hadoop, HBase, Drill, ML) • Applied research (IE/AT) • Nowadays mainly Go (Python, Node, PHP, Java, C++) • Dev turned ops $ whois mhausenblas
  3. Hit me up on Twitter: @mhausenblas 3 data scientist data

    engineer developer SRE/admin architect PM PHB
  4. Hit me up on Twitter: @mhausenblas 7 Challenges • interchanging

    models • versioning of datasets and models • building apps: integrating ML features • deployments (local vs. at scale, GPU support) data scientists data engineers developers ops
  5. Hit me up on Twitter: @mhausenblas 9 • Dotmesh via

    
 https://dotmesh.com/try-dotmesh/ • Kubeflow via
 https://www.katacoda.com/kubeflow/scenarios/deploying-kubeflow Demo
  6. Hit me up on Twitter: @mhausenblas 11 Kubernetes kubernetes.io •

    Container lifecycle management • Declarative, state-driven • Extensible, modular API • Robust, flexible, scalable Kudos to Lucas Käldström for this figure (source)
  7. Hit me up on Twitter: @mhausenblas 12 Kubeflow github.com/kubeflow/kubeflow •

    Launched in late 2017 by Google • JupyterHub • TensorFlow Training Controller and Server • Intel, Red Hat + growing community
  8. Hit me up on Twitter: @mhausenblas 13 Pachyderm pachyderm.io •

    Graph-oriented data pipeline • Version control • Clients for Python, Go, Scala, etc.
  9. Hit me up on Twitter: @mhausenblas 14 Binder mybinder.org •

    Turns a GitHub repo with Jupyter notebooks into interactive notebooks using Docker • Serves via a JupyterHub server
  10. Hit me up on Twitter: @mhausenblas 15 Dotmesh dotmesh.com •

    Data state management across microservices • Operating on a filesystem level • Externalize snapshotting • Troubleshooting, debugging
  11. Hit me up on Twitter: @mhausenblas 18 Engage! • https://kube-machine-learning.rocks

    • https://github.com/gaocegege/kubeflow-weekly • Kubernetes Machine Learning WG • https://github.com/kubernetes/community/tree/master/wg-machine-learning • https://groups.google.com/forum/#!forum/kubernetes-wg-machine-learning • OpenShift Machine Learning SIG • https://commons.openshift.org/sig/OpenshiftMachineLearning.html
  12. Hit me up on Twitter: @mhausenblas 19 Learn! • https://developers.google.com/machine-learning/crash-course/

    • https://github.com/Sarasra/models • https://js.tensorflow.org/ • https://learn.openshift.com