Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Challenge of Automatically Configuring Nume...

regonn
April 08, 2023

The Challenge of Automatically Configuring Numerai Compute Environment

I have tried to use Terraform to build the Numerai Compute environment so that it can be built automatically.

regonn

April 08, 2023
Tweet

More Decks by regonn

Other Decks in Technology

Transcript

  1. Regonn (@regonn_haizine) • Shimane, Japan • Freelancer(Machine Learning Engineer, Metaverse,

    NeuroTech) • Orca(Solana Chain DEX) Japanese community manager • Farmer My past glory
  2. Numerai Compute • ࣗಈͰNumeraiʹαϒϛοτ͢Δ࢓૊Έ • Webhook URL Λొ࿥͢Δ • Daily

    submission͕͍ͨ͠ • numerai-cli ͩͱ AWS Fargate Λར༻ • https://pypi.org/project/numerai-cli/ • ࠓճ͸ࣗ෼Ͱͦͷ؀ڥΛߏஙͯ͠ΈΔ
  3. Google Cloud Functions with Terraform • Fargateͱಉ͡Α͏ʹGPU͸࢖͑ͳ͍ • αʔόʔϨεͰॲཧΛ࣮ߦՄೳ •

    ϝϞϦ͸࠷େ16GB(Preview 32GB) ※ϑΝΠϧ༰ྔ΋ΠϯϝϞϦͳͨΊ஫ҙ • GCFଆ͸ PythonϑΝΠϧ(main.py)ͱґଘϥΠϒϥϦఆٛ(requirements.txt) • Terraform ͰΠϯϑϥߏங༻ͷϑΝΠϧΛఆٛ • https://github.com/regonn/numerai-gcf-terraform
  4. Google Cloud Run with Terraform • ϝϞϦ͕࠷େ 32 GB(Cloud functions

    ͸࠷େ16GB ※Preview: 32GB) • ࣮ߦ࣌ؒ͸ಉ͘͡࠷େ60෼ • ͨͩ͠ɺFastAPI౳ͷΤϯυϙΠϯτΛར༻ͨ͠DockerΛ࡞੒͢Δඞཁ͕͋Δ • Cloud functions Ͱ͸ରԠݴޠͷ੍ݶ͕͋Δ͕ɺCloud Run Ͱ͸ࣗ෼ͷ޷͖ͳ ݴޠΛબ୒Ͱ͖Δ