Upgrade to Pro — share decks privately, control downloads, hide ads and more …

経済学者に知ってほしい機械学習 ~反事実モデルによる予測~ / JEA2020 tutori...

経済学者に知ってほしい機械学習 ~反事実モデルによる予測~ / JEA2020 tutorial CFML

日経学会2020年度春季大会の企画セッションでの発表内容になります。

https://www.jeameetings.org/2020s/index.html

Kazuki Taniguchi

May 30, 2020
Tweet

More Decks by Kazuki Taniguchi

Other Decks in Science

Transcript

  1. • ৬ྺ • 2014.4-2019.3 • גࣜձࣾαΠόʔΤʔδΣϯτΞυςΫຊ෦ AI Lab • 2019.4-

    • ITܥϕϯνϟʔ (ϓϩμΫτ։ൃ/ϚʔέςΟϯά) • ϑϦʔϥϯε (AI/MLͷݚڀ։ൃ) • ݚڀ෼໺ • Pattern Recognition / Image Restoration • Recommendation / Response Prediction • Counterfactual ML ࣗݾ঺հ ୩ޱ ࿨ً (@kazk1018)
  2. (લஔ͖) ػցֶशͷఆٛ • ҰൠతͳػցֶशͷఆٛΑΓ΋ڱٛ f(x) .PEFM Input: Կ͔͠Βͷσʔλ (਺஋, ը૾,

    ςΩετ, etc…) Output: ग़ྗ݁Ռ (ϥϕϧ, ਺஋, ը૾, etc…) ༧ଌ ਪఆ
  3. ϥʔϝϯ࣍࿠ͷళฮΛAutoMLͰ൑ผ  • 1170ຕ × 41ళฮ = ໿48,000ຕͷը૾ΛϥϕϧͱηοτͰ༻ҙ • Google

    AutoML VisionΛར༻ͯ͠95%ͷਫ਼౓Λ࣮ݱ • ػցֶशͷ஌͕ࣝͳ͍ਓؒͰ΋ֶश࣌ؒ͸18෼ͰϞσϧΛ֫ಘ
  4. ػցֶशͷࠜຊతͳ՝୊ • ༧ଌ͍ͨ͠σʔλͷ෼෍ͷఆٛ (Ϟσϧͷཁ݅) • ʮHorseʯ • ʮHorse on the

    grassʯ • ʮHorse Toyʯ • બ୒όΠΞε • ར༻͢ΔαϯϓϧΛબ୒͢Δ࣌఺Ͱൃੜ͢ΔόΠΞε • ֶश͢Δσʔλֶ͕श͢Δσʔλͦͷ΋ͷʹґଘͯ͠બ୒͞Ε͍ͯ Δ৔߹ʹൃੜ
  5. ޿ࠂ഑৴γεςϜ • Real Time Bidding (RTB) imp click Request RTB

    click͕؍ଌͰ͖ͳ͍ = ϥϕϧ͕ଘࡏ͠ͳ͍ win = ޿ࠂ͕දࣔ(imp)͞ΕΔ lose = ޿ࠂ͕දࣔ(imp)͞Εͳ͍ SSP DSP
  6. ޿ࠂ഑৴γεςϜ • ΫϦοΫ༧ଌϞσϧ [2] *% ޿ࠂ Ϣʔβ JNQ DMJDL 

    " B :FT   # E /P  $ F /P  # C :FT   $ F :FT  ഑৴γεςϜͷϩάσʔλ
  7. ޿ࠂ഑৴γεςϜ • ΫϦοΫ༧ଌϞσϧ [2] *% ޿ࠂ Ϣʔβ JNQ DMJDL 

    " B :FT   # E /P  $ F /P  # C :FT   $ F :FT  ഑৴γεςϜͷϩάσʔλ ֶशʹར༻Ͱ͖Δσʔλ
  8. ޿ࠂ഑৴γεςϜͷ໰୊ઃఆ • ΫϦοΫ༧ଌϞσϧ [2] *% ޿ࠂ Ϣʔβ JNQ DMJDL 

    " B :FT   # E /P  $ F /P  # C :FT   $ F :FT  ഑৴γεςϜͷϩάσʔλ ࣮ࡍʹΫϦοΫΛ༧ଌ͍ͨ͠σʔλ ൓ࣄ࣮
  9. ڞมྔγϑτ [3] • ҎԼͷ৚݅ͷ໰୊Λѻ͏ p(x) ≠ p′(x) p(y|x) = p′(y|x)

    ͸ ʹಠཱʹै͍, ͸ ʹ ಠཱʹै͏ͱԾఆ͢Δ D = {(xi , yi )}n i=1 p(x, y) D′ = {x′ i }m i=1 ∫ p′(x, y)dy ͜ͷͱ͖, ࣍ͷΑ͏ͳ৚݅Λຬͨ͢ͱ͖Λڞมྔγϑτͱ͍͏
  10. ڞมྔγϑτԼͷ༧ଌϞσϧ • ͱ Λ༻͍ͯ৽ͨͳೖྗ ʹର͢Δऔಘ Λ ༧ଌ͢ΔϞσϧ Λֶश͍ͨ͠ • ଛࣦؔ਺Λ

    ͱ͢Δͱ͖ڞมྔγϑτ͸ॏཁ౓ॏΈ෇ ͚Λ༻͍Δ͜ͱͰղܾ͢Δ͜ͱ͕஌ΒΕ͍ͯΔ [3] {(xi , yi )}n i=0 {x′ j }m j=0 x′ y fθ (x) loss(y, fθ ) minθ n ∑ i=0 w(xi )loss(yi , fθ (xi ))) w(xi ) = p′(xi ) p(xi )
  11. ॏཁ౓ॏΈ෇͚Λ༻͍ͨΫϦοΫ༧ଌ P(x) ≠ P′(x) ※ ͸RTBͰ ͕͖ͨͱ͖ͷwin͢Δ৚݅෇͖֬཰ q(win|x) x ΛԾఆ͢Δͱ,


    ॏཁ౓ॏΈ෇͚ʹΑͬͯ༧ଌϞσϧΛֶश͢Δ͜ͱ͕Մೳ p(y|x) = p′(y|x) P(x) = q(win|x)P′(x) P′(x) ༧ଌ͍ͨ͠σʔλͷ֬཰෼෍: ֶशσʔλͷ֬཰෼෍:
  12. Unsupervised Domain Adaptation • Domain Adaptation (υϝΠϯదԠ) • े෼ͳ৘ใͷ͋ΔυϝΠϯ(Source Domain)ͷ஌ࣝΛ৘ใ͕গͳ͍


    υϝΠϯ(Target Domain)ʹదԠ͢Δٕज़ • సҠֶशͷҰछ • Unsupervised Domain Adaptation • Target Domainʹڭࢣϥϕϧ͕ͳ͍υϝΠϯదԠ
  13. Domain Adaptation Neural Network (DANN) [5] labelͷ༧ଌʹ͓͚Δଛࣦ domainͷ༧ଌʹ͓͚Δଛࣦ ( ̂

    θy , ̂ θf ) = arg min θy ,θf L(y, d, x) ̂ θd = arg max θd L(y, d, x) ಛ௃දݱ ͸υϝΠϯͷݟ෼͚͕͔ͭͳ͘ͳΔΑ͏ʹ ϥϕϧͷ༧ଌʹ͓͚ΔଛࣦΛ࠷খԽ͢Δ f
  14. CTR Prediction by using DANN [6] • ࣮ݧ • σʔληοτ

    • Criteo CTR Prediction Contest (Kaggle) • ൺֱख๏ • Baseline: Deep Neural Network (Only source dataset) • Importance Sampling
  15. ·ͱΊ • ػցֶशͷٸ଎ͳීٴͱظ଴ • ExplainabilityͱStability • ػցֶशͷ՝୊ • ޿ࠂ഑৴γεςϜͷࣄྫ •

    ޿ࠂ഑৴γεςϜͷ֓ཁ • ൓ࣄ࣮Λߟྀͨ͠ػցֶशʹΑΔΫϦοΫ༧ଌ • ڞมྔγϑτ • Unsupervised Domain Adaptation
  16. References 1. Kun Kuang, Peng Cui, Susan Athey, Ruoxuan Xiong,

    and Bo Li, “Stable Prediction across Unknown Environments”, KDD, 2018 2. Oliver Chapelle, Eren Manavoglu, Romer Rosales, “Simple and scalable response prediction for display advertising”, TIST, 2015 3. Hidetoshi Shimodaira, “Improving predictive inference under covariate shift by weighting the log-likelihood function”, JSPI, 2000 4. James M. Robins, Andrea Rotnitzky, Lue Ping Zhao, “Estimation of Regression Coefficients When Some Regressors Are Not Always Observed”, JASA, 1994 5. Yaroslav Ganin, Victor Lempitsky, “Unsupervised Domain Adaptation by Backpropagation”, JMLR, 2015 6. ୩ޱ ࿨ً, ҆Ҫ ᠳଠ, “Domain Adaptation Neural NetworksΛ༻͍ͨΫϦοΫ༧ଌ”, JSAI, 2019