Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Offline A/B testing for Recommender Systems

alpicola
November 20, 2018

Offline A/B testing for Recommender Systems

alpicola

November 20, 2018
Tweet

More Decks by alpicola

Other Decks in Technology

Transcript

  1. Offline A/B testing for Recommender Systems — CriteoͷWSDM'18ͷ࿦จ — SpotifyͷRecSys'18࿦จͰ΋ݴٴ

    — ΫοΫύου։࠵ͷಡΈձͰ͢Ͱʹ঺հ͞Ε͍ͯͨ — ͕ɺվΊͯ۷ΓԼ͛ͨ࿩͕Ͱ͖Ε͹ͱࢥ͍·͢ 3
  2. ैདྷख๏ — Importance sampling (IS) — Normalized importance sampling (NIS)

    — Doubly robust estimator (DR) — Capped importance sampling (CIS) — Normalized capped importance sampling (NCIS) ౷ܭ΍ϞϯςΧϧϩ๏ͷจ຺Ͱొ৔ 10
  3. Importance sampling (IS) — ! όΠΞε͕ͳ͍ — — " ʹΑΔߴόϦΞϯε

    (unbounded) — όϦΞϯε͕େ͖͍ͱ ͱ ΛൺֱͰ͖ͳ͍ 11
  4. Midzuno-Sen method 1. Λαϯϓϧ 2. Λ ͔Β ͳ΋ͷ͕ಘΒΕΔ·Ͱαϯϓϧ 3. Λ

    ͔Βαϯϓϧ 4. Λฦ͢ ͜͏ͯ͠ಘΒΕΔ஋Λ ͱॻ͘ 31
  5. ࣮ݧ݁Ռͷ·ͱΊ — CIS͸૬͕ؔෛ — શମతʹ௿Ίͷਪఆ͕ग़͍ͯͨ (Figure 4) — CIS⇒NCISͰେ͖͘վળ —

    NCIS⇒PointNCISͰِཅੑ཰͕͞ΒʹԼ͕Δ — ద߹཰͸NCISҎޙͦ͜·ͰΑ͘ͳΒͳ͍ — ࣮ߦ଎౓ʹ͓͍ͯ΋ਫ਼౓ʹ͓͍ͯ΋PointNCIS͕Α͍ 36