Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ScaleShift-jp-2019-summer
Search
ryo nakamaru
July 17, 2019
Research
1
210
ScaleShift-jp-2019-summer
ScaleShift の機能概要、Kubernetes クラスタとの連携について
ryo nakamaru
July 17, 2019
Tweet
Share
More Decks by ryo nakamaru
See All by ryo nakamaru
AWSで楽をするサービスメッシュ入門/appmesh-trial
pottava
1
1.5k
reinforce-2019-recap-lt
pottava
2
4.1k
Firecracker とは何か/what is Firecracker
pottava
12
5.5k
ハイブリッド並列 on Kubernetes/hybrid-parallel-program-on-kubernetes
pottava
1
440
AWS Fargate + Code 兄弟で始める継続的デリバリー / Continuous Delivery with AWS Fargate and Code brothers
pottava
12
3.2k
Singularity と NVIDIA GPU Cloud で作る ハイブリッド機械学習環境の構築 / Building a hybrid environment for Machine Learning with Singularity and NGC
pottava
3
1.4k
明日から始めるちょい足し λ / get-started-with-aws-lambda
pottava
4
2.5k
NGC と Singularity によるハイブリッド機械学習環境 / A hybrid environment for Machine Learning with NGC and Singularity
pottava
0
500
NGC × Singularity での機械学習環境/MachineLearning environment with NGC and Singularity
pottava
2
800
Other Decks in Research
See All in Research
長期・短期メモリを活用したエージェントの個別最適化
isidaitc
0
140
EOGS: Gaussian Splatting for Efficient Satellite Image Photogrammetry
satai
4
610
EarthSynth: Generating Informative Earth Observation with Diffusion Models
satai
3
340
AI in Enterprises - Java and Open Source to the Rescue
ivargrimstad
0
320
国際論文を出そう!ICRA / IROS / RA-L への論文投稿の心構えとノウハウ / RSJ2025 Luncheon Seminar
koide3
8
5.2k
電通総研の生成AI・エージェントの取り組みエンジニアリング業務向けAI活用事例紹介
isidaitc
1
1k
20250605_新交通システム推進議連_熊本都市圏「車1割削減、渋滞半減、公共交通2倍」から考える地方都市交通政策
trafficbrain
0
820
Minimax and Bayes Optimal Best-arm Identification: Adaptive Experimental Design for Treatment Choice
masakat0
0
180
SSII2025 [SS1] レンズレスカメラ
ssii
PRO
2
1.1k
SSII2025 [TS1] 光学・物理原理に基づく深層画像生成
ssii
PRO
4
4.3k
集合間Bregmanダイバージェンスと置換不変NNによるその学習
wasyro
0
150
投資戦略202508
pw
0
560
Featured
See All Featured
Art, The Web, and Tiny UX
lynnandtonic
303
21k
Gamification - CAS2011
davidbonilla
81
5.4k
Six Lessons from altMBA
skipperchong
28
4k
Learning to Love Humans: Emotional Interface Design
aarron
274
40k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
30
9.7k
Into the Great Unknown - MozCon
thekraken
40
2.1k
Why Our Code Smells
bkeepers
PRO
339
57k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
140
34k
Rails Girls Zürich Keynote
gr2m
95
14k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
36
2.5k
We Have a Design System, Now What?
morganepeng
53
7.8k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Transcript
ScaleShift ΦϯϓϨϛε / Ϋϥυ Ͱ࣮ݱ͢Δػցֶशڥ June, 2019
!2 ScaleShift Docker ϕʔεɺΦʔϓϯιʔεͷ Web ΫϥΠΞϯτ ΞϓϦέʔγϣϯͰ͢ • ϞσϧߏஙϑΣʔζ -
NGC / ࣗࣾϦϙδτϦ͔Βػցֶश Docker ΠϝʔδΛϫϯΫϦοΫͰऔಘ - ͦͷҙͷ Docker ΠϝʔδΛ Jupyter notebook ίϯςφͱͯ͠ىಈ • ϞσϧֶशϑΣʔζ - ߏஙʹར༻ͨ͠ϥΠϒϥϦ͝ͱ Docker ΠϝʔδʹݻΊϦϙδτϦอଘ - ΫϦοΫ͚ͩͰ Kubernetes Ϋϥελ / Rescale େنܭࢉλεΫΛૹ৴
جຊతͳಈ͖ !3 How does it work?
!4 ScaleShift ͷىಈ ϩʔΧϧʹ Web αʔόʔ্ཱ͕͕ͪΓ·͢
!5 ػցֶशιϑτΣΞͷΠϯετʔϧ NGC / ϓϥΠϕʔτϨδετϦ ͔ΒϫϯΫϦοΫͰμϯϩʔυ
!6 Jupyter notebook ͰͷϞσϧߏங Jupyter Ͱϥοϓͨ͠ίϯςφ͕͔ΜͨΜʹىಈ ϙʔτ࡞ۀྖҬ ίϯςφ͝ͱʹ ͞Εͨ ΫϦʔϯͳڥ
ɹ.
!7 େنܭࢉͷͨΊͷϥοϐϯά ґଘϥΠϒϥϦιʔείʔυ܈Λ·ͱΊɺͻͱͭͷΠϝʔδʹݻΊ·͢
!8 ࣾΫϥελ / ΫϥυܭࢉλεΫೖ ೖઌʹԠͯ͡ඞཁͳ API ͕࣮ߦ͞Ε·͢ ར༻ϦιʔεྔΛܾΊ ΫϥελλεΫೖ
Kubernetes ࿈ܞ !9 Integration with a kubernetes cluster
!10 ػցֶश ͱ Kubernetes Web ք۾Λத৺ʹίϯςφΦʔέετϨʔγϣϯͷσϑΝΫτʹͳͬͨ k8sɻ ػցֶशͷจ຺Ͱίϯςφར༻͕ΜʹͳΓɺԠ༻ࣄྫ͕૿͍͑ͯ·͢ɻ - NVIDIA
͕ެࣜʹαϙʔτΛද໌ [ GTC 2018 Keynote, March 27 ] - Mercari ML Ops Night Vol.1 [ גࣜձࣾ ϝϧΧϦ / May 23, 2018 ] ɹhttps://mercari.connpass.com/event/85931/presentation/ - Jupyter ͚ͩͰػցֶशΛ࣮αʔϏεల։Ͱ͖Δج൫ [ גࣜձࣾϦΫϧʔτϥΠϑελΠϧ ] ɹhttps://engineer.recruit-lifestyle.co.jp/techblog/2018-10-04-ml-platform/ - KubernetesʹΑΔػցֶशج൫ͷઓ [ גࣜձࣾ Preferred Networks / Dec 4, 2018 ] ɹhttps://www.slideshare.net/pfi/kubernetes-125013757
!11 ScaleShift + Kubernetes ߏྫ ετϨʔδ ཧϊʔυ ܭࢉϊʔυ ࣾωοτϫʔΫ NGC
DockerHub ϓϥΠϕʔτ ϨδετϦ Kubernetes ݚڀ / ։ൃνʔϜ ScaleShift ೖΓ ϩʔΧϧ
!12 1. ػցֶशιϑτΣΞͷબ ετϨʔδ ཧϊʔυ ܭࢉϊʔυ ࣾωοτϫʔΫ NGC DockerHub ϓϥΠϕʔτ
ϨδετϦ Kubernetes ݚڀ / ։ൃνʔϜ GUI ͔ΒબͿ͚ͩͰ μϯϩʔυ͕࢝·Γ·͢
!13 2. Ϟσϧߏங ετϨʔδ ཧϊʔυ ܭࢉϊʔυ ࣾωοτϫʔΫ NGC DockerHub ϓϥΠϕʔτ
ϨδετϦ Kubernetes ݚڀ / ։ൃνʔϜ ScaleShift ͕ ϊʔτϒοΫΛىಈ͠·͢
!14 3. ࣮ߦڥɾೖྗσʔλͷసૹ ετϨʔδ ཧϊʔυ ܭࢉϊʔυ ࣾωοτϫʔΫ NGC DockerHub ϓϥΠϕʔτ
ϨδετϦ Kubernetes ScaleShift ͕෦తʹ ඞཁͳసૹΛߦ͍·͢ ݚڀ / ։ൃνʔϜ
!15 4. େنܭࢉͷ࣮ߦΛࢦࣔ ετϨʔδ ཧϊʔυ ܭࢉϊʔυ ࣾωοτϫʔΫ NGC DockerHub ϓϥΠϕʔτ
ϨδετϦ Kubernetes ݚڀ / ։ൃνʔϜ Kubernetes ͷ Job ͱͯ͠ ܭࢉ݅Λૹ৴͠·͢
!16 5. େنܭࢉͷ࣮ߦ ετϨʔδ ཧϊʔυ ܭࢉϊʔυ ࣾωοτϫʔΫ NGC DockerHub Kubernetes
ϓϥΠϕʔτ ϨδετϦ ݚڀ / ։ൃνʔϜ
!17 6. ܭࢉ݁Ռͷ֬ೝ ετϨʔδ ཧϊʔυ ܭࢉϊʔυ ࣾωοτϫʔΫ NGC DockerHub ϓϥΠϕʔτ
ϨδετϦ Kubernetes ݚڀ / ։ൃνʔϜ
!18 Kubernetes ઃఆ / λεΫ࣮ߦը໘
ScaleShift ͷઃఆ !19 Configurations
!20 ֎෦࿈ܞ ࿈ܞػೳ ઃఆ NVIDIA GPU CLOUD • NVIDIA ࣾͷཧ͢Δػցֶश
Docker Πϝʔδͷ Ұཡ / ৄࡉใऔಘɺΠϝʔδͷμϯϩʔυ ɹAPI Ωʔ & Ϣʔβઃఆ ϓϥΠϕʔτϨδετϦ • ࣗࣾͰཧ͢Δػցֶश Docker Πϝʔδͷ ҰཡใऔಘɺΠϝʔδͷμϯϩʔυ ɹଓઌ & Ϣʔβઃఆ AWS • ػցֶश Docker Πϝʔδͷμϯϩʔυ • ϩʔΧϧϑΝΠϧγεςϜͱ S3 ؒͷσʔλ࿈ܞ ʢ࣮༧ఆʣ Kubernetes • ࣾΫϥελ / ΫϥυͰͷେنܭࢉ࣮ߦ ɹkubecfg Rescale • Rescale ϓϥοτϑΥʔϜͰͷେنܭࢉ࣮ߦ ɹҬࢦఆ & API Ωʔ
!21 ىಈΦϓγϣϯʢൈਮʣ ઃఆ֓ཁ ॳظ SS_JUPYTER_MINIMUM_PORT ίϯςφͷଓϙʔτಈతׂ։࢝൪߸ ɾ30000 SS_LOG_LEVEL ΞϓϦέʔγϣϯͷϩάग़ྗϨϕϧ ɹwarn
SS_WORKSPACE_HOST_DIR ϗετଆͷ࡞ۀσʔλอଘྖҬ ɹͳ͠ʢࢦఆඞਢʣ SS_NGC_REGISTRY_ENDPOINT NGC ଓઌ ɹhttps://registry.nvidia.com SS_NGC_REGISTRY_USER_NAME NGC Ϣʔβʔ໊ ɹ$oauthtoken SS_RESCALE_SINGULARITY_VERSION Rescale Ͱͷ Singularity ϥϯλΠϜόʔδϣϯ ɹ3.2.0 SS_RESCALE_JOB_WALLTIME Rescale ͰͷλεΫ࣮ߦ࠷େ࣌ؒ ɾ3600 docker-compose.yml ʹઃఆΛهࡌɺىಈͰ͖·͢