Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ScaleShift-jp-2019-summer

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for ryo nakamaru ryo nakamaru
July 17, 2019
Research
220
1

 ScaleShift-jp-2019-summer

ScaleShift の機能概要、Kubernetes クラスタとの連携について

Avatar for ryo nakamaru

ryo nakamaru

July 17, 2019

More Decks by ryo nakamaru

See All by ryo nakamaru
AWSで楽をするサービスメッシュ入門/appmesh-trial
Avatar for ryo nakamaru pottava
1
1.6k
reinforce-2019-recap-lt
Avatar for ryo nakamaru pottava
2
4.1k
Firecracker とは何か/what is Firecracker
Avatar for ryo nakamaru pottava
12
5.6k
ハイブリッド並列 on Kubernetes/hybrid-parallel-program-on-kubernetes
Avatar for ryo nakamaru pottava
1
460
AWS Fargate + Code 兄弟で始める継続的デリバリー / Continuous Delivery with AWS Fargate and Code brothers
Avatar for ryo nakamaru pottava
12
3.3k
Singularity と NVIDIA GPU Cloud で作る ハイブリッド機械学習環境の構築 / Building a hybrid environment for Machine Learning with Singularity and NGC
Avatar for ryo nakamaru pottava
3
1.5k
明日から始めるちょい足し λ / get-started-with-aws-lambda
Avatar for ryo nakamaru pottava
4
2.5k
NGC と Singularity によるハイブリッド機械学習環境 / A hybrid environment for Machine Learning with NGC and Singularity
Avatar for ryo nakamaru pottava
0
520
NGC × Singularity での機械学習環境/MachineLearning environment with NGC and Singularity
Avatar for ryo nakamaru pottava
2
870

Other Decks in Research

See All in Research
IEEE AIxVR 2026 Keynote Talk: "Beyond Visibility: Understanding Scenes and Humans under Challenging Conditions with Diverse Sensing"
Avatar for Mariko Isogawa miso2024
0
150
通時的な類似度行列に基づく単語の意味変化の分析
Avatar for hajime kiyama rudorudo11
0
250
ScoreMatchingRiesz for Automatic Debiased Machine Learning and Policy Path Estimation with an Application to Japanese Monetary Policy Evaluation
Avatar for MasaKat0 masakat0
0
240
湯村研究室の紹介2025 / yumulab2025
Avatar for yumulab yumulab
0
330
明日から使える！研究効率化ツール入門
Avatar for Yusuke Matsui matsui_528
11
6k
英語教育 “研究” のあり方：学術知とアウトリーチの緊張関係
Avatar for TerasawaT terasawat
1
800
論文紹介 "ReSim: Reliable World Simulation for Autonomous Driving"
Avatar for Takuma Kogo kogo
0
120
老舗ものづくり企業でリサーチが変革を起こすまで - 三菱重工DXの実践
Avatar for kazu skydats
0
120
The Landscape of Agentic Reinforcement Learning for LLMs: A Survey
Avatar for Shunsuke KITADA shunk031
4
760
"主観で終わらせない"定性データ活用 ― プロダクトディスカバリーを加速させるインサイトマネジメント / Utilizing qualitative data that "doesn't end with subjectivity" - Insight management that accelerates product discovery
Avatar for 株式会社カミナシ kaminashi
16
24k
量子コンピュータの紹介
Avatar for OQTOPUS oqtopus
0
260
[SITA2025 Workshop] 空中計算による高速・低遅延な分散回帰分析
Avatar for Koya SATO k_sato
0
140

Featured

See All Featured
Navigating Team Friction
Avatar for Lara Hogan lara
192
16k
How To Stay Up To Date on Web Technology
Avatar for Chris Coyier chriscoyier
790
250k
Fight the Zombie Pattern Library - RWD Summit 2016
Avatar for Marcelo Somers marcelosomers
234
17k
YesSQL, Process and Tooling at Scale
Avatar for Rocio Delgado rocio
174
15k
Measuring & Analyzing Core Web Vitals
Avatar for Philip Tellis bluesmoon
9
810
Organizational Design Perspectives: An Ontology of Organizational Design Elements
Avatar for Dr. Kim W Petersen kimpetersen
PRO
1
670
Mozcon NYC 2025: Stop Losing SEO Traffic
Avatar for Sam Torres samtorres
0
200
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
Avatar for Aki / @LoveIdahoBurger aki_iinuma
128
55k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
Avatar for Dan Newman danielanewman
231
23k
Design of three-dimensional binary manipulators for pick-and-place task avoiding obstacles (IECON2024)
Avatar for konakalab konakalab
0
400
Groundhog Day: Seeking Process in Gaming for Health
Avatar for Sebastian Deterding codingconduct
0
140
The World Runs on Bad Software
Avatar for Brandon Keepers bkeepers
PRO
72
12k

Transcript

  1. ScaleShift ΦϯϓϨϛε / Ϋϥ΢υ Ͱ࣮ݱ͢Δػցֶश؀ڥ June, 2019

  2. !2 ScaleShift Docker ϕʔεɺΦʔϓϯιʔεͷ Web ΫϥΠΞϯτ ΞϓϦέʔγϣϯͰ͢ • ϞσϧߏஙϑΣʔζ -

    NGC / ࣗࣾϦϙδτϦ͔Βػցֶश Docker ΠϝʔδΛϫϯΫϦοΫͰऔಘ - ͦͷ೚ҙͷ Docker ΠϝʔδΛ Jupyter notebook ίϯςφͱͯ͠ىಈ • ϞσϧֶशϑΣʔζ - ߏஙʹར༻ͨ͠ϥΠϒϥϦ͝ͱ Docker ΠϝʔδʹݻΊϦϙδτϦ΁อଘ - ΫϦοΫ͚ͩͰ Kubernetes Ϋϥελ / Rescale ΁େن໛ܭࢉλεΫΛૹ৴

  3. جຊతͳಈ͖ !3 How does it work?

  4. !4 ScaleShift ͷىಈ ϩʔΧϧʹ Web αʔόʔ্ཱ͕͕ͪΓ·͢

  5. !5 ػցֶशιϑτ΢ΣΞͷΠϯετʔϧ NGC / ϓϥΠϕʔτϨδετϦ ͔ΒϫϯΫϦοΫͰμ΢ϯϩʔυ

  6. !6 Jupyter notebook ͰͷϞσϧߏங Jupyter Ͱϥοϓͨ͠ίϯςφ͕͔ΜͨΜʹىಈ ϙʔτ΋࡞ۀྖҬ΋ ίϯςφ͝ͱʹ ෼཭͞Εͨ ΫϦʔϯͳ؀ڥ

    ɹ.

  7. !7 େن໛ܭࢉͷͨΊͷϥοϐϯά ґଘϥΠϒϥϦ΍ιʔείʔυ܈Λ·ͱΊɺͻͱͭͷΠϝʔδʹݻΊ·͢

  8. !8 ࣾ಺Ϋϥελ / Ϋϥ΢υ΁ܭࢉλεΫ౤ೖ ౤ೖઌʹԠͯ͡ඞཁͳ API ͕࣮ߦ͞Ε·͢ ར༻ϦιʔεྔΛܾΊ Ϋϥελ΁λεΫ౤ೖ

  9. Kubernetes ࿈ܞ !9 Integration with a kubernetes cluster

  10. !10 ػցֶश ͱ Kubernetes Web ք۾Λத৺ʹίϯςφΦʔέετϨʔγϣϯͷσϑΝΫτʹͳͬͨ k8sɻ ػցֶशͷจ຺Ͱ΋ίϯςφར༻͕੝ΜʹͳΓɺԠ༻ࣄྫ͕૿͍͑ͯ·͢ɻ - NVIDIA

    ͕ެࣜʹαϙʔτΛද໌ [ GTC 2018 Keynote, March 27 ] - Mercari ML Ops Night Vol.1 [ גࣜձࣾ ϝϧΧϦ / May 23, 2018 ] ɹhttps://mercari.connpass.com/event/85931/presentation/ - Jupyter ͚ͩͰػցֶशΛ࣮αʔϏεల։Ͱ͖Δج൫ [ גࣜձࣾϦΫϧʔτϥΠϑελΠϧ ] ɹhttps://engineer.recruit-lifestyle.co.jp/techblog/2018-10-04-ml-platform/ - KubernetesʹΑΔػցֶशج൫΁ͷ௅ઓ [ גࣜձࣾ Preferred Networks / Dec 4, 2018 ] ɹhttps://www.slideshare.net/pfi/kubernetes-125013757

  11. !11 ScaleShift + Kubernetes ߏ੒ྫ ετϨʔδ ؅ཧϊʔυ ܭࢉϊʔυ ࣾ಺ωοτϫʔΫ NGC

    DockerHub ϓϥΠϕʔτ ϨδετϦ Kubernetes ݚڀ / ։ൃνʔϜ ScaleShift ೖΓ ϩʔΧϧ୺຤

  12. !12 1. ػցֶशιϑτ΢ΣΞͷબ୒ ετϨʔδ ؅ཧϊʔυ ܭࢉϊʔυ ࣾ಺ωοτϫʔΫ NGC DockerHub ϓϥΠϕʔτ

    ϨδετϦ Kubernetes ݚڀ / ։ൃνʔϜ GUI ͔ΒબͿ͚ͩͰ μ΢ϯϩʔυ͕࢝·Γ·͢

  13. !13 2. Ϟσϧߏங ετϨʔδ ؅ཧϊʔυ ܭࢉϊʔυ ࣾ಺ωοτϫʔΫ NGC DockerHub ϓϥΠϕʔτ

    ϨδετϦ Kubernetes ݚڀ / ։ൃνʔϜ ScaleShift ͕ ϊʔτϒοΫΛىಈ͠·͢

  14. !14 3. ࣮ߦ؀ڥɾೖྗσʔλͷసૹ ετϨʔδ ؅ཧϊʔυ ܭࢉϊʔυ ࣾ಺ωοτϫʔΫ NGC DockerHub ϓϥΠϕʔτ

    ϨδετϦ Kubernetes ScaleShift ͕಺෦తʹ ඞཁͳసૹΛߦ͍·͢ ݚڀ / ։ൃνʔϜ

  15. !15 4. େن໛ܭࢉͷ࣮ߦΛࢦࣔ ετϨʔδ ؅ཧϊʔυ ܭࢉϊʔυ ࣾ಺ωοτϫʔΫ NGC DockerHub ϓϥΠϕʔτ

    ϨδετϦ Kubernetes ݚڀ / ։ൃνʔϜ Kubernetes ͷ Job ͱͯ͠ ܭࢉ৚݅Λૹ৴͠·͢

  16. !16 5. େن໛ܭࢉͷ࣮ߦ ετϨʔδ ؅ཧϊʔυ ܭࢉϊʔυ ࣾ಺ωοτϫʔΫ NGC DockerHub Kubernetes

    ϓϥΠϕʔτ ϨδετϦ ݚڀ / ։ൃνʔϜ

  17. !17 6. ܭࢉ݁Ռͷ֬ೝ ετϨʔδ ؅ཧϊʔυ ܭࢉϊʔυ ࣾ಺ωοτϫʔΫ NGC DockerHub ϓϥΠϕʔτ

    ϨδετϦ Kubernetes ݚڀ / ։ൃνʔϜ

  18. !18 Kubernetes ઃఆ / λεΫ࣮ߦը໘

  19. ScaleShift ͷઃఆ !19 Configurations

  20. !20 ֎෦࿈ܞ ࿈ܞػೳ ઃఆ஋ NVIDIA GPU CLOUD • NVIDIA ࣾͷ؅ཧ͢Δػցֶश

    Docker Πϝʔδͷ Ұཡ / ৄࡉ৘ใऔಘɺΠϝʔδͷμ΢ϯϩʔυ ɹAPI Ωʔ & Ϣʔβઃఆ ϓϥΠϕʔτϨδετϦ • ࣗࣾͰ؅ཧ͢Δػցֶश Docker Πϝʔδͷ Ұཡ৘ใऔಘɺΠϝʔδͷμ΢ϯϩʔυ ɹ઀ଓઌ & Ϣʔβઃఆ AWS • ػցֶश Docker Πϝʔδͷμ΢ϯϩʔυ • ϩʔΧϧϑΝΠϧγεςϜͱ S3 ؒͷσʔλ࿈ܞ ʢ࣮૷༧ఆʣ Kubernetes • ࣾ಺Ϋϥελ / Ϋϥ΢υͰͷେن໛ܭࢉ࣮ߦ ɹkubecfg Rescale • Rescale ϓϥοτϑΥʔϜͰͷେن໛ܭࢉ࣮ߦ ɹ஍Ҭࢦఆ & API Ωʔ

  21. !21 ىಈΦϓγϣϯʢൈਮʣ ઃఆ֓ཁ ॳظ஋ SS_JUPYTER_MINIMUM_PORT ίϯςφ΁ͷ઀ଓϙʔτಈతׂ౰։࢝൪߸ ɾ30000 SS_LOG_LEVEL ΞϓϦέʔγϣϯͷϩάग़ྗϨϕϧ ɹwarn

    SS_WORKSPACE_HOST_DIR ϗετଆͷ࡞ۀσʔλอଘྖҬ ɹͳ͠ʢࢦఆඞਢʣ SS_NGC_REGISTRY_ENDPOINT NGC ઀ଓઌ ɹhttps://registry.nvidia.com SS_NGC_REGISTRY_USER_NAME NGC Ϣʔβʔ໊ ɹ$oauthtoken SS_RESCALE_SINGULARITY_VERSION Rescale Ͱͷ Singularity ϥϯλΠϜόʔδϣϯ ɹ3.2.0 SS_RESCALE_JOB_WALLTIME Rescale ͰͷλεΫ࣮ߦ࠷େ࣌ؒ ɾ3600 docker-compose.yml ʹઃఆΛهࡌɺىಈͰ͖·͢