Share
Embed
CFMLの概要と研究動向 / cfml #1 introduction
CFML勉強会#1の資料です。(https://connpass.com/event/128714/)
Kazuki Taniguchi
July 10, 2019
Transcript
28 slides
-
• ৬ྺ • 2014.4-2019.3 • גࣜձࣾαΠόʔΤʔδΣϯτ ΞυςΫຊ෦ AI Lab •
2019.4- • ͷελʔτΞοϓ • ϑϦʔϥϯε(AI/MLͷݚڀ։ൃ) • ݚڀ • Pattern Recognition / Image Super Resolution • Recommendation / Response Prediction • Counterfactual ML ࣗݾհ ୩ޱ ً (@kazk1018) -
Summary • Counterfactual Machine Learningͷ֓ཁ • ࣄ࣮͕ੜ͡ΔσʔλΛ༻͍ͨػցֶशͰ͋Δ • Interactive LearningͱCausal
Inference͕ڞมྔγϑτͷ෦ Ͱ͋Δ͜ͱΛઆ໌͢Δ • ݚڀಈ • ࠂͷίϯϖʹΑͬͯσʔληοτ͕ެ։͞Ε͍ͯΔ • ֶ֤ձͷWorkshopΛհ͢Δ -
ઃఆ ͋ΔϢʔβ͕ਪનͨ͠ΛΫϦοΫ(Feedback)͔ͨ͠ Ͳ͏͔ͷϩάΛऔಘ͍ͯ͠Δ ࣌ࠁ Ϣʔβ ΫϦοΫ " 9
/P # : /P $ 9 :FT # ; /P $ ; /P աڈϩά -
Online Evaluation Randomized Controlled Experiment (A/B Testing) • ਖ਼֬ͳൺֱΛߦ͏͜ͱ͕Մೳ (Gold
Standard) • ݁Ռ͕ग़Δ·Ͱʹ͕͔͔࣌ؒΔ • ৽͍͠ํࡦͷόάUXԼͷϦεΫ͕͋Δ • ຊ൪ಋೖͷ։ൃίετ͕େ͖͍ 50% 50% π π0 -
Offline Evaluation • طଘͷϩάσʔλ͔Β৽͍͠PolicyΛධՁ͢Δ • ࣮ࡍʹݟ͍ͤͯͳ͍Λબ͢Δ߹ධՁ͕Ͱ ͖ͳ͍ ࣌ࠁ Ϣʔβ
ΫϦοΫ " 9 /P → BΛਪનͨ͠߹ʁ π(x) ΫϦοΫ͞ΕΔ͔Ͳ͏͔Θ͔Βͳ͍ (Counterfactual) -
Counterfactual Machine Learning ࣄ࣮͕ੜ͡ΔσʔλΛ༻͍ͨػցֶश Causal Inference Interactive Learning Counterfactual Machine
Learning ※͜ͷൃදͰڞมྔγϑτͷ؍͔Βݟ͍ͯ͘ (From A Machine Learning Perspective) -
ڞมྔγϑτ • ҎԼͷ݅ͷΛѻ͏ p(x) ≠ p′(x) p(y|x) = p′(y|x)
ʹಠཱʹै͍, ʹ ಠཱʹै͏ͱԾఆ͢Δ D = {(xi , yi )}n i=1 p(x, y) D′ = {x′ i }m i=1 ∫ p′(x, y)dy ͜ͷͱ͖, ࣍ͷΑ͏ͳ݅Λຬͨ͢ͱ͖Λڞมྔγϑτͱ͍͏ -
ڞมྔγϑτ • ྫ) Իೝࣝ, ը૾ೝࣝ, etc… ɹ : Իσʔλ, :
ऀ, : ࢠڙ, : େਓ ɹ : ը૾σʔλ, : ਓ, : ࣨ, : ֎ x y p(x) p′(x) x y p(x) p′(x) p(x, y) = p(y|x)p(x) p′(x, y) = p(y|x)p′(x) x y y′ x′ -
ڞมྔγϑτԼͷ༧ଌϞσϧ • ͱ Λ༻͍ͯ৽ͨͳೖྗ ʹର͢Δऔಘ Λ ༧ଌ͢ΔϞσϧ Λֶश͍ͨ͠ • ଛࣦؔΛ
ͱ͢Δͱ͖ڞมྔγϑτॏཁॏΈ ͖Λ༻͍Δ͜ͱͰղܾ͢Δ͜ͱ͕ΒΕ͍ͯΔ {(xi , yi )}n i=0 {x′}m i=0 x′ y fθ (x) loss(y, fθ ) minθ n ∑ i=0 w(xi )loss(yi , fθ (xi ))) w(xi ) = p′(xi ) p(xi ) -
ڞมྔγϑτԼͷ༧ଌϞσϧ • ূ໌ Ep′[loss(y, fθ (x))] = ∫ ∫ loss(y,
fθ (x))p′(x, y)dxdy = ∫ ∫ loss(y, fθ (x))p′(x)p′(y|x)dxdy = ∫ ∫ loss(y, fθ (x))p′(x)p′(y|x) p(x) p(x) dxdy = ∫ ∫ loss(y, fθ (x))p(x)p(y|x) p′(x) p(x) dxdy = ∫ ∫ loss(y, fθ (x))p(x, y)w(x)dxdy ≈ n ∑ i=0 loss(y, fθ (x))w(x) -
Interactive Learning Context: x Policy: π(x) Action: a = π(x)
Reward: δ(x, a) System a User $POUFYU "DUJPO 3FXBSE 5JNF 6TFS9 " /P 5JNF 6TFS: # /P 5JNF 6TFS9 $ :FT 5JNF 6TFS; # /P 5JNF 6TFS: $ /P Logging ᶃ ᶄ ᶅ ᶆ x ∼ P(x) -
Interactive Learning 3FDPNNFOEBUJPO $POUFYUVBM CBOEJU 3FJOGPSDFNFOU -FBSOJOH $POUFYU 6TFSBOE*UFN *OGPSNBUJPO
$POUFYU 4UBUF "DUJPO *UFN *% "SN "DUJPO 3FXBSE $MJDL 1VSDIBTF 3FXBSE 3FXBSE (ৄ͘͠੪౻ͷൃදͰ) -
Causal Inference • ͋ΔༀΛױऀʹ༩͢Δ͔Ͳ͏͔Ͱ3ޙʹපؾ͕࣏ͬ ͍ͯΔ͔Ͳ͏͔ͷҼՌޮՌΛଌΓ͍ͨ ☓ ױऀ: x Treatment: t
පؾ͕࣏Δ͔Ͳ͏͔: y ༀΛ༩͍ͯ͠ͳ͍߹ ༀΛ༩ͨ͠߹ -
Causal Inference • ࣮ࡍʹ؍ଌ͢ΔͷͲͪΒ͔Ұํ͚ͩͰ͋Δ ☓ ױऀ: x Treatment: t පؾ͕࣏Δ͔Ͳ͏͔:
y ҼՌਪͷࠜຊ ࣄ࣮ (ৄ҆͘͠ҪͷൃදͰ) ༀΛ༩͍ͯ͠ͳ͍߹ ༀΛ༩ͨ͠߹ -
ڞมྔγϑτͱͷؔ • Interactive Learning • ͷੜաఔʹ؍ଌ͕͋Δ͔Ͳ͏͔ͷҧ͍ • ϢʔβͷinteractionʹΑΔ؍ଌͷ༗ແͰੜ·ΕΔγϑτ D p(x)
= q(o|x)p′(x) w(xi ) = 1 q(o|xi ) -
ڞมྔγϑτͱͷؔ • Causal Inference • ͷੜաఔʹtreatment͕͋Δ͔Ͳ͏͔ͷҧ͍ • ዞҙతͳtreatmentͷׂʹΑͬͯੜ·ΕΔγϑτ D p(x)
= q(t|x)P(x) p′(x) = q(¬t|x)P(x) w(x) = q(¬t|x) q(t|x) (=Propensity Score) -
• CounterfactualΛѻͬͨػցֶशͷCompetition • ੈք࠷େͷDSPͰ͋ΔCriteo͕σʔληοτΛެ։ • ࠂͷஔͷ༧ଌͰΫϦοΫͷ࠷େԽ • CounterfactualΛߟྀͨ͠ධՁࢦඪΛಋೖ Criteo Ad
Placement Challenge https://www.crowdai.org/challenges/nips-17-workshop-criteo-ad-placement-challenge -
Workshops • KDD • Causal Discovery, 2018/2019 • Offline and
Online Evaluation of Interactive Systems, 2019 • NIPS • From ‘What if?’ To ‘What next?’, 2017 • Causal Learning, 2018 • RecSys • REVEAL, 2018/2019 • ICML • FAIM’18 Workshop(CausalML) -
Summary • Counterfactual Machine Learningͷ֓ཁ • ࣄ࣮͕ੜ͡ΔσʔλΛ༻͍ͨػցֶशͰ͋Δ • Interactive LearningͱCausal
Inference͕ڞมྔγϑτͷ෦ Ͱ͋Δ͜ͱΛઆ໌ͨ͠ • ݚڀಈ • ࠂͷίϯϖʹΑͬͯσʔληοτ͕ެ։͞Ε͍ͯΔ • ֶ֤ձͷWorkshopΛհ͢Δ -
References 1. Home Page of Thorsten Joachims http://www.cs.cornell.edu/people/tj/ 2. Counterfactual
Reasoning and Learning Systems: The Example of Computational Advertising, Leon Bottou, et al…, JMLR 2013 https://www.microsoft.com/en-us/research/wp-content/uploads/2013/11/ bottou13a.pdf 3. ඇఆৗڥԼͰͷֶशɿڞมྔγϑτదԠɼ ΫϥεόϥϯεมԽదԠɼมԽݕ http://www.ms.k.u-tokyo.ac.jp/2014/NonstationarityReview-jp.pdf 4. Offline Evaluation and Optimization for Interactive Systems https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/ tutorial.pdf Πϥετ by ͔Θ͍͍ϑϦʔૉࡐू ͍Β͢ͱ