Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Shiva 〜Nextremerをscale upする機械学習環境〜

Chanmoro
December 14, 2017

Shiva 〜Nextremerをscale upする機械学習環境〜

Nextremer 社内で運用している機械学習環境 Shiva のご紹介
tag: Docker, Mesos, Marathon, Jupyter, ceph,

Chanmoro

December 14, 2017
Tweet

More Decks by Chanmoro

Other Decks in Technology

Transcript

  1. Copyright © Nextremer, inc. All Rights Reserved. 2 ࣗݾ঺հ ྆֯

    ࿨ي(΋ΖͣΈ ͔͖ͣ) גࣜձࣾNextremer ϓϩάϥϚʔ ※ର࿩γεςϜͷݚڀ։ൃΛ͍ͯ͠ΔاۀͰ͢ • ओʹ΍͍ͬͯΔ͜ͱ • αʔόʔαΠυ։ൃ • ͜͜࠷ۙ৮ٕͬͨज़ • Terraform, Ansible, Packer ← Infrasctucture as Codeʂ • Docker, Kubernetes, Apache Mesos ← ϙʔλϏϦςΟ͍͜͞͏ʂ • Python, Scala, javascript(ES6) ← ؔ਺ܕ͏·͔͚͘Δͱؾ͍͍࣋ͪʂ • Finagle, React.js ← ϚΠΫϩαʔϏε͍ͨ͠ʂϦΞΫςΟϒ͍ͨ͠ʂ
  2. Copyright © Nextremer, inc. All Rights Reserved. 4 ػցֶश؀ڥ Shiva

    1. ΦϑΟεʹ͋ΔGPUϚγϯʹ 2. WebͷμογϡϘʔυ͔Β 3. ઐ༻ͷJupyterΛ্ཱͪ͛ΒΕΔ
  3. Copyright © Nextremer, inc. All Rights Reserved. 5 Shiva͕ղܾ͢Δ՝୊ ▪

    GPUϚγϯΛ࢖͍͍͚ͨͲɾɾɾ • Ϋϥ΢υඅ༻Λؾʹͯ͠ؾܰʹ࢖͑ͳ͍ • ओʹ༧ࢉ෇͖ͷPJҎ֎ɺτϨʔχϯάͱ͔ɺҋݚڀͱ͔ • GPUϚγϯΛෳ਺ϢʔβʔͰڞ༗͢Δͷ΋΍Γʹ͍͘ • ϥΠϒϥϦͷڝ߹ɺ୭͕࢖ͬͯΔ͔Θ͔ΒΜ • GPUϚγϯ্Ͱͷ؀ڥߏங͕େมͩͬͨΓ • શһ͕ৄ͍͠Θ͚͡Όͳ͍
  4. Copyright © Nextremer, inc. All Rights Reserved. 7 Abstract ▪

    ϢʔβʔຖͷDockerίϯςφ ▪ શ෦ೖΓ؀ڥ & sudoݖݶ ▪ ϗʔϜdirΛશϚγϯͰڞ༗
  5. Copyright © Nextremer, inc. All Rights Reserved. 9 Architecture Overview

    μογϡϘʔυ͔ΒAPIܦ༝Ͱૢ࡞ ɹɾϦιʔεར༻ঢ়گΛऔಘ ɹɾίϯςφͷىಈɾ࡟আ
  6. Copyright © Nextremer, inc. All Rights Reserved. 10 Architecture Overview

    શ୆ͰϗʔϜσΟϨΫτϦΛڞ༗ ɹɾ෼ࢄετϨʔδΛߏங ɹɾίϯςφىಈ࣌ʹϚ΢ϯτ
  7. Copyright © Nextremer, inc. All Rights Reserved. 11 Architecture Overview

    ▪ ϢʔβʔͱϙʔτͷϚοϐϯά • http://host/user/kazuki.morozumi • Jupyter͕Ϧεϯ͢Δϙʔτ͸ϥϯμϜ ▪ ίϯςφՔಇঢ়گΛެ։͢ΔAPI ▪ tmpnbΛΧελϚΠζ • https://github.com/jupyter/tmpnb
  8. Copyright © Nextremer, inc. All Rights Reserved. 13 Infrastructure Overview

    ίϯςφΦʔέετϨʔγϣϯʹ͸ Mesos + MarathonΛར༻
  9. Copyright © Nextremer, inc. All Rights Reserved. 14 Infrastructure Overview

    ▪ Mesos+MarathonΛબ୒ͨ͠ཧ༝ • ಛఆͷΞϓϦΛσʔϞϯͱͯ͠ಈ͔ͤΕ͹े෼ • αʔϏεσΟεΧόϦͱ͔͸͍Βͳ͍ • Kubernetes͸ΦʔόʔεϖοΫ͗͢Δ • ߏ੒ཁૉ͕ଟ͘ෳࡶͰߏஙɺϝϯςφϯε͕େม • ୯७ʹMesos+MarathonΛࢼͯ͠Έ͔ͨͬͨ • ߏங౰࣌͸৘ใ͕͔ͳΓগͳ͔ͬͨ
  10. Copyright © Nextremer, inc. All Rights Reserved. 15 Infrastructure Overview

    CPU Intel Core i7-6700K ϝϞϦ 32GB Disk SSD 500GB GPU GTX-1080 ×2ຕ GPUϚγϯεϖοΫ ݟͨ໨΋େࣄʂ ɹ- ͱΓ͋͑ͣޫΒͤΔ ɹ- ΈΜͳʹݟ͑Δ৔ॴʹஔ͘
  11. Copyright © Nextremer, inc. All Rights Reserved. 16 ӡ༻͍ͯͯ͠ىͬͨ͜໰୊ ▪

    σΟεΫ্ݶ • ֶशσʔλ਺ඦGBΛΞοϓϩʔυͨ͠ΒσΟεΫ্ݶʹୡͯ͠ఀࢭ • ίϯςφଆ͔ΒsshfsͰࣗ෼ͷPCͷσΟεΫΛϚ΢ϯτͯ͠ճආͨ͠ ▪ ίϯςφ͕๫૸ͨ࣌͠ͷηʔϑςΟʔωοτ • ։ൃͨ͠ίʔυͷόάͰϗετͷCPUɾϝϞϦΛ৯͍௵͢ • ϏΪφʔͷϢʔβʔ͕͜ͷΑ͏ͳόάΛ࡞Γ͕ͪ • ରԠࡦ͸ίϯςφຖʹϦιʔε੍ݶΛ͚ͭΔ • ස౓௿͔ͭӨڹ௿ͳͷͰಛʹݱঢ়͸ରԠͤͣ
  12. Copyright © Nextremer, inc. All Rights Reserved. 20 NextremerͰ͸͜Μͳ͜ͱΛ΍͍ͬͯ·͢ ▪

    ฐࣾCEO ޲ҪΛίϐʔͨ͠ର࿩γεςϜ • 2017/10 ୈ8ճର࿩γεςϜγϯϙδ΢ϜʹͯσϞൃද