Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Istio Meets ML Ops - ML@Loft#9

Lain Matsuoka
December 19, 2019

Istio Meets ML Ops - ML@Loft#9

Lain Matsuoka

December 19, 2019
Tweet

More Decks by Lain Matsuoka

Other Decks in Programming

Transcript

  1. 2 ! ॴଐɿAIνʔϜ ιϑτ΢ΣΞΤϯδχΞ ◦ 2019/01ʙ ! ϝϧΧϦͰ΍ͬͨϓϩδΣΫτʢൈਮʣ ◦ Ձ֨αδΣετ

    ◦ τϨʔσΟϯάΧʔυը૾ೝࣝ ◦ ݕࡧϦϥϯΩϯάʢόοΫΤϯυʣ ◦ ΦϯϥΠϯ࣮ݧ؀ڥߏஙʢόοΫΤϯυʣ ! ϞσϦϯά͔ΒόοΫΤϯυɺΠϯϑϥ։ൃ· Ͱ෯޿͘୲౰ Lain Matsuoka/দԬ ྰԻ
  2. 6 ݱߦͷػցֶशαʔϏεͨͪ ! ҧ൓ग़඼ݕ஌ʢෳ਺υϝΠϯʣ = API + TF Serving !

    Ձ֨αδΣετ = API + TF Serving ! ײಈग़඼ʢ৘ใͷࣗಈิ׬ʣ = API + TF/ONNX Serving + Faiss ! ഑ૹํ๏αδΣετ = API + PostgreSQL ! ը૾ݕࡧ ɹ = API + TF Serving + Faiss ! etc. ଟ༷ͳΞʔΩςΫνϟͷਪ࿦αʔϏεΛKubernetesΫϥελʔ্Ͱӡ༻
  3. 7 ػցֶशαʔϏεͷಛੑ ! ୯७ʹܭࢉෛՙ͕ߴ͍ ! ෳ਺ίϯϙʔωϯτΛ૊Έ߹ΘͤͨෳࡶͳΞʔΩςΫνϟʹͳΓ΍͍͢ ◦ લॲཧɺޙॲཧΛߦ͏APIαʔόʔ ◦ Deep

    Learningͷਪ࿦αʔόʔʢTF Serving, ONNXRuntimeʣ ◦ ྨࣅ౓ݕࡧαʔόʔʢFaiss, NMSLIBʣ ◦ छʑͷσʔλϕʔεʢRedis, PostgreSQLʣ IstioͰαʔϏεϝογϡΛߏங͠ɺޮ཰తʹ؅ཧ
  4. 9 ϚΠΫϩαʔϏεͱαʔϏεϝογϡ Control Plane Data Plane Service A Service Container

    Envoy Proxy Service B Service Container Envoy Proxy Service C Service Container Envoy Proxy Service D Service Container Envoy Proxy ! ϚΠΫϩαʔϏε܈ͷӡ༻ͱ࿈ܞΛޮ཰Խ͢Δ
  5. 13 ෛՙ΍ϨΠςϯγͷϞχλϦϯάɺࣗಈεέʔϦϯά ! Envoy͕औಘ͢ΔϝτϦΫεΛPrometheus + GrafanaͰ؂ࢹ ◦ ඵؒϦΫΤετ਺ ◦ ϨΠςϯγ

    ◦ CPUɾϝϞϦར༻཰ ! औಘͨ͠ϝτϦΫεΛݩʹɺHPAʢHorizontal Pod AutoscalerʣͰࣗಈε έʔϦϯά
  6. 14 ΧφϦΞσϓϩΠʹΑΔθϩμ΢ϯλΠϜϦϦʔε ! IstioͷVirtual ServiceʹΑΔτϥϑΟοΫ੍ޚΛ࢖༻ ◦ KubernetesͷServiceͷΈͰ͸ෆՄೳͳཻ౓Ͱ੍ޚՄೳ ! ࠶ֶशͨ͠Ϟσϧ΍ɺ৽نϑΟʔνϟʔΛ҆શʹϦϦʔεͰ͖Δ ◦

    ྲྀྔΛ0ʹ͢Δ͚ͩͰ҆શʹϩʔϧόοΫ - route: - destination: host: service-a.default.svc.cluster.local
 weight: 99 - destination: host: service-b.default.svc.cluster.local
 weight: 1 Service A Service B Virtual Service Service A Service B Virtual Service
  7. 15 ϛϥʔϦϯάʹΑΔຊ൪؀ڥͰͷαʔϏεݕূ ! IstioͷVirtual Service͸ϛϥʔϦϯάػೳ΋αϙʔτ ◦ ΦϦδφϧͷτϥϑΟοΫͷ׬શͳίϐʔΛྲྀೖͤ͞ΒΕΔ ! ࣮؀ڥͰͷϦΫΤετͰ҆શʹಈ࡞ݕূͰ͖Δ Service

    A Service B Virtual Service - route: - destination: host: service-a.default.svc.cluster.local
 weight: 100 mirror: host: service-b.default.svc.cluster.local
 mirror_percent: 100