Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
KubeflowではじめるML on Kubernetes #1 Kubeflowの概要とセッ...
Search
Keita Watanabe
August 17, 2022
1
460
KubeflowではじめるML on Kubernetes #1 Kubeflowの概要とセットアップ
Keita Watanabe
August 17, 2022
Tweet
Share
More Decks by Keita Watanabe
See All by Keita Watanabe
Scalable Infrastructure for Large-Scale AI Training with AWS Sagemaker Hyperpod @ Singapore AI Hour
keitaw
0
3
[AWS Summit Japan 2025] Optimizing Foundation Model Development with Amazon SageMaker HyperPod: Insights from Training the Amazon Nova Model
keitaw
0
13
Building foundation models on AWS
keitaw
0
170
[re:Invent2024 Chalktalk] Cost-effectively deploy PyTorch LLMs on AWS Inferentia using Amazon EKS
keitaw
0
71
AWS Summit New York 2024: CMP 301 Demystifying the ML software stack on Amazon EC2 accelerated instances
keitaw
0
320
re:Invent 2023 CMP319 Deploy LLMs with AWS Inferentia & Ray to optimize performance and cost
keitaw
0
32
re:Invent 2023: CMP332 De-mystifying ML software stack on Amazon EC2 accelerated instances
keitaw
0
140
AWS における LLM・GenAI 大規模学習への取り組み / Large scale GenAI・LLM training on AWS
keitaw
1
530
Amazon EC2 シリコン革命 / Amazon EC2 Silicon Innovation
keitaw
0
180
Featured
See All Featured
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Building Applications with DynamoDB
mza
95
6.5k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3.1k
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
Code Reviewing Like a Champion
maltzj
524
40k
Building Adaptive Systems
keathley
43
2.7k
jQuery: Nuts, Bolts and Bling
dougneiner
63
7.8k
The Cost Of JavaScript in 2023
addyosmani
51
8.6k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
126
53k
Why You Should Never Use an ORM
jnunemaker
PRO
58
9.4k
How GitHub (no longer) Works
holman
314
140k
Transcript
Keita Watanabe Kubernetes Meetup Tokyo # 52 LT KubeflowͰ͡ΊΔ ML
on Kubernetes #1 Kube fl owͷ֓ཁͱηοτΞοϓ
Keita Watanabe Machine Learning Solutions Architect ▶ Self-managedͳMachine LearningͷWorkload Λ୲͍ͯ͠ΔSA
▶ લ৬ͰDatascientist/ML Researcherͱͯ͠ ECαΠτ্ͷը૾ݕࡧػೳͷ։ൃʹैࣄ ▶ Twitter: keitaw09 ▶ Linkedin: keitawatanabe ▶ ݱॴଐ: AWS Japan αϯσΟΤΰͰग़ձͬͨτϦ झຯ
Kubeflowͱ Machine Learning Model-Development Lifecycle (MDLC)ͷ ֤εςοϓʢ+ύΠϓϥΠϯʣΛΧόʔ͢ΔToolͷू·Γ KubeflowΛ࠾༻͢ΔϝϦοτ Composability: ༻్ʹԠͯ͡ඞཁͳComponentsΛΓग़ͯ͠༻͍Δ͜ͱ͕Ͱ͖Δ
Portability: Vendor lock-inΛආ͚Δ͜ͱ͕Ͱ͖Δ Scalability: ෛՙʹԠͯ͡Container/NodeΛεέʔϧ͢Δ͜ͱ͕Ͱ͖Δ
Kubeflow overview Central Dashboard Data Ingestion Data Analysis Data Transform
Data Validation Training AutoML Model eval. and valid. Serving Logging KServe Seldon Kube fl ow Pipelines Kube fl ow Notebooks Katib Training Operators MDLCͷ֤εςοϓ ରԠ͢ΔTool
Kubeflow overview Central Dashboard Data Ingestion Data Analysis Data Transform
Data Validation Training AutoML Model eval. and valid. Serving Logging KServe Seldon Kube fl ow Pipelines Kube fl ow Notebooks Katib Training Operators MDLCͷ֤εςοϓ ରԠ͢ΔTool ࠓΔ͜ͱ: Kube fl owͷηοτΞοϓ
Kubeflow setup Amazon EKS্ͰͷKube fl owڥͷߏங • git • kubectl
• eksctl • kustomize • ࠓճAWS Cloud9্Ͱ࡞ۀ͠·͢ cf. https://www.eksworkshop.com/ 020_prerequisites/workspace/ Prerequisites
Demo
Kubeflow setup
To be
࣍ճ…
ࢀߟࢿྉ • https://www.eksworkshop.com/020_prerequisites/workspace/ • https://www.kube fl ow.org/docs/components/ • https://awslabs.github.io/kube fl
ow-manifests/docs/deployment/vanilla/guide/ • https://www.oreilly.com/library/view/kube fl ow-for-machine/9781492050117/