Upgrade to Pro — share decks privately, control downloads, hide ads and more …

user-behaviour-vol1

KARAKURI Inc.
November 16, 2021

 user-behaviour-vol1

ユーザー行動予測に関する研究のサーベイ

KARAKURI Inc.

November 16, 2021
Tweet

More Decks by KARAKURI Inc.

Other Decks in Research

Transcript

  1. ྑ͍ධՁࢦඪͱ͸ʁ 7 • SensitivityɿվળΛͪΌΜͱݕ஌Ͱ͖Δ͔ʁ • Trustworthinessɿ݁Ռ͸৴པͰ͖Δ͔ʁ • Efficiencyɿܭࢉɾܭଌͷίετ͕௿͍͔ʁ • Debuggability

    and ActionabilityɿࢦඪͷมԽͷཧ༝ͷಛఆ͕Մೳ͔ʁ • Interpretability and DirectionalityɿࢦඪͷมԽ͸໨తΛୡ੒͢Δ͔ʁ [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]
  2. Overall Evaluation Criteria (OEC) 8 • ࠷ऴతͳ໨తͱͳΔࢦඪ • DirectivityͱSensitivity͕ॏཁ •

    ͜ΕΒ2ͭΛຬͨ͢ͷ͸ࠔ೉ • ྫɿ1Ϣʔβ͋ͨΓͷฏۉऩӹ [Dmitriev+ KDD 2017] [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]
  3. ͦͷଞͷࢦඪ 9 • Guardrail Metricsɿ੍໿৚݅ • Operational Metricsɿσόοά༻ͷࢦඪ • Data

    Quality Metricsɿσʔλ͕৴པͰ͖Δ͜ͱΛอূ͢Δࢦඪ [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]
  4. Ϛ΢ευϥοάɾϖʔδεΫϩʔϧ 12 ɾυϥοάͱεΫϩʔϧʹҙਤ͕ग़Δ ɾεϚϗͰ͸ಘΒΕͳ͍ ୅ΘΓʹεϫΠϓ ॎυϥοά ԣυϥοά υϥοά͕஗͍ ؔ࿈ੑ ×

    ◦ ◦ εΫϩʔϧྔ εΫϩʔϧස౓ εΫϩʔϧ଎౓ ؔ࿈ੑ × ◦ ◦ [Guo & Agichtein WWW 2012, Arapakis & Leiva SIGIR 2020] [Guo+ SIGIR 2013, Lagun+ SIGIR 2014] ݕࡧ݁Ռͷจॻ͕ཉ͍͠৘ใͱؔ܎ͯ͠ ͍Δ͔͕Χʔιϧͷಈ͖͔ΒΘ͔Δ
  5. ଺ࡏ࣌ؒʢDwell Timeʣ 13 [Yi+ RecSys 2014], [Lu+ WWW 2019] •

    ଺ࡏ࣌ؒ͸ϢʔβʔҙਤΛΑ͘൓ө͍ͯ͠Δ • UIʹґଘ͢Δ • ΦϑϥΠϯͷ܇࿅࣌ʹ͔͠࢖͑ͳ͍ [Lalmas + KDD 2015] [Ouyang+ KDD 2019] %XFMM5JNF 8FCQBHF [Yom-Tovi+ BigData 2013], [Yi+ RecSys 2014] • ϖʔδ΍σόΠε͝ͱʹ͹Βͭ͘
  6. ϢʔβʔΤϯήʔδϝϯτͷࢦඪ 19 • Click Depth IndexɿϖʔδϏϡʔ • Duration IndexɿαΠτ଺ࡏ࣌ؒ •

    Interaction IndexɿߪೖɾΞοϓϩʔυ • Recency Indexɿස౓ • Loyalty Indexɿ௕ظؒ [Tutorial on Online User Engagement KDD 2020]
  7. Ϣʔβʔຬ଍౓༧ଌ 21 • Jointly Leveraging Intent and Interaction Signals to

    Predict User Satisfaction with Slate Recommendations [Mehrotra+ WWW 2019] • 16ͷϢʔβʔߦಈ͔ΒϢʔβʔຬ଍౓Λ༧ଌʢSpotifyʣ • ΠϯλϏϡʔɼαʔϕΠɼϊϯύϥϕΠζͰϢʔβʔͷҙਤΛಉఆ • ϢʔβʔҙਤΛߟྀͨ͠΄͏͕ΑΓΑ͘Ϣʔβʔຬ଍౓Λ༧ଌͰ͖ͨ • Ϣʔβʔҙਤ͝ͱʹॏཁͱͳΔϢʔβʔߦಈ͕͹Βͭ͘
  8. Ϣʔβʔͷߪങ཭୤༧ଌ 23 • Predicting Shopping Behavior with Mixture of RNNs

    [Toh+ SIGIR 2017] • ߪങ͢Δ͔൱͔ɼ·ͨ͸Ӿཡ͍ͯ͠Δ͚͔ͩΛ༧ଌʢָఱʣ • ΫϦοΫετϦʔϜσʔλɹ→ɹ଺ࡏ࣌ؒͱϖʔδλΠϓͷϖΞͷྻ • RNNͰߴ͍ਫ਼౓Ͱ༧ଌͰ͖Δ͜ͱΛ֬ೝ • ߪങ͢Δ͔൱͔ͷ൑அ͸Ӿཡ͍ͯ͠Δ͔൱͔ͷ൑அΑΓ೉͍͠
  9. ໨తࢥߟͳϢʔβʔͷ෼ੳɾ༧ଌ 27 • Predicting Intent Using Activity Logs: How Goal

    Specificity and Temporal Range Affect User Behavior [Cheng+ WWW 2017] • PinterestϢʔβʔ͕໨తࢥߟ͔ɼͦΕ͕ߦಈ͔Β༧ଌͰ͖Δ͔ • ໨తࢤ޲ͷ৔߹ɼݕࡧ͕ଟ͘ෳࡶͰɼݕࡧʹ͔͔Δ·Ͱ͕ૣ͘ɼݟΔ ίϯςϯπ͸গͳ͍͕ΑΓ۩ମతͰɼը૾ͷιʔεʹඈͿ܏޲͕͋Δ • Ӿཡɼ֦େɼΫϦοΫɼݕࡧͳͲͷߦಈ͔ΒϥϯμϜϑΥϨετͰ Ϣʔβʔ͕໨తࢥߟ͔Λ༧ଌ • ݕࡧ͕༧ଌʹॏཁͰɼ࠷ॳͷ෼ͷߦಈͰ໨తࢥߟ͔͕༧ଌͰ͖ͨ
  10. νϟʔϯϨʔτ༧ଌ 29 • I Know You’ll Be Back: Interpretable New

    User Clustering and Churn Prediction on a Mobile Social Application [Yang+ KDD 2018] • SnapϢʔβʔͷνϟʔϯϨʔτͷ༧ଌ • ϢʔβʔΛΫϥελϦϯάͰηάϝϯτΘ͚ͯ͠཭୤཰͕ߴ͍Ϣʔ βʔΛಛఆ͠ɼAttention͖ͭLSTMͰνϟʔϯϨʔτΛ༧ଌ
  11. Ϣʔβʔͷ࣭໰ͷ༧ଌ 35 • Reinforcement Learning for User Intent Prediction in

    Customer Service Bots [Chen+ SIGIR 2019] • νϟοτϘοτͷͨΊͷਪનΞϧΰϦζϜʢAnt financialʣ • ࣭໰͞ΕΔલʹߦಈཤྺ͔Β࣭໰Λ༧ଌ͠ɼ࣭໰ީิΛఏҊ ʢUser Intent Predictionʣ
  12. ख๏ 37 • CTR༧ଌ • ϢʔβʔͷΫϦοΫܥྻͱϢʔβʔ৘ใΛೖྗͱͯ͠࢖༻ • લऀ͸CNNɼޙऀ͸FCNNͰಛ௃ྔΛ࡞੒͠ɼͦΕΒΛFCNN΁ೖྗ • Question

    popularity • աڈҰఆظؒʹ࣭໰͕ΫϦοΫ͞Εͨઈର਺ͱදࣔ͞Εͨճ਺͔Βɼ ࠓΑ͘ฉ͔Ε͍ͯΔ࣭໰Λࢉग़ • Question Diversity • ݱࡏͷ࣭໰ͱੲͷ࣭໰ͷྨࣅ౓͔Β࣭໰ͷଟ༷ੑΛܭࢉ • ͜ΕΒͷείΞΛ࢖ͬͯtop-Nͷ࣭໰ީิΛϥϯΫ͚ͮ
  13. ࣮ࡍʹγεςϜͱͯ͠ӡ༻͞Ε͍ͯΔ 39 • AntProphet: an Intention Mining System behind Alipay’s

    Intelligent Customer Service Bot [Chen+ IJCAI 2020 (demo)] • Ϣʔβʔͷ࣭໰ͷʹରԠ • ଟ͘ͷ࣭໰͕ϢʔβʔߦಈͷΈ͔Β ਪଌͰ͖Δ͜ͱΛ֬ೝ
  14. ϢʔβʔߦಈܥྻΛ׆༻ͨ͠CTR༧ଌ 40 • Deep Interest Network for Click-Through Rate Prediction

    [Zhou + KDD 2018] • AttentionΛ༻͍Δ͜ͱͰϢʔβʔͷΞΠςϜ΁ͷؔ৺Λදݱͨ͠ CTR༧ଌʢAlibabaʣ
  15. ϢʔβʔߦಈܥྻΛ׆༻ͨ͠CTR༧ଌ 44 • User Behavior Retrieval for Click-Through Rate Prediction

    [Qin + SIGIR 2020] • ௕͍ϢʔβʔߦಈܥྻΛ׆༻͢ΔCTR༧ଌʢAlibabaʣ • AttentionϕʔεͷωοτϫʔΫͰੲͷߦಈཤྺ͔Βݕࡧ • ͋Δจ຺Ͱ࠷΋͋Γ͏ΔUser-ItemͷϖΞΛ༧ଌ
  16. ϢʔβʔߦಈܥྻΛ׆༻ͨ͠CTR༧ଌ 47 • Practice on Long Sequential User Behavior Modeling

    for Click- Through Rate Prediction [Pi+ KDD 2019] • ௕͍ϢʔβʔߦಈܥྻΛར༻ͨ͠ΦϯϥΠϯ޿ࠂCTR༧ଌͷվળ ʢAlibabaʣ • Neural Turing MachineΛར༻͠storageͷ੍໿ͱlatencyͷ੍໿ʹରԠ
  17. ద੾ͳλΠϛϯάͰద੾ͳ΋ͷΛਪન 50 • Temporal-Contextual Recommendation in Real-Time [Ma+ KDD 2020]

    • ֊૚తͳRNNΛ࢖͏͜ͱͰҙਤͷมԽΛଊ͑ͯɼద੾ͳλΠϛϯάͰద ੾ͳ΋ͷΛਪન͢ΔϞσϧʢAmazonʣ
  18. ϢʔβʔͷߪങߦಈΛཅʹϞσϦϯά 53 • Opportunity Models for E-commerce Recommendation: Right Product,

    Right Time [Wang & Zhang SIGIR 2013] • ੜଘ࣌ؒ෼ੳʹΑΓ͋Δ࣌ؒͰ͋Δ঎඼ΛϢʔβʔ͕ങ͏֬཰Λදݱ • shop.comͰͷߪങߦಈͷ༧ଌʹ੒ޭ͠CVRͱϢʔβʔຬ଍౓΋޲্
  19. ύʔνΣεϑΝωϧΛҙࣝͨ͠Ϟσϧ 54 • Understanding Consumer Journey using Attention based Recurrent

    Neural Networks [Zhou + KDD 2019] • ύʔνΣεϑΝωϧͷͲ͜ʹ͍Δ͔Λֶश͢Δattention͖ͭRNNʢYahooʣ
  20. ϩδεςΟοΫճؼʁ 55 ɾSimple and Scalable Response Prediction for Display Advertising

    [Chapelle+ 2014] (Criteo) ɾAd Click Prediction: a View from the Trenches [McMahan + KDD 2013] (Google) ɾϚΠΫϩΞυʹ͓͚ΔCTR༧ଌ΁ͷऔΓ૊Έ<MicroAd>
  21. ͦͷଞ 61 • Time-Aware Prospective Modeling of Users for Online

    Display Advertising [Gligorijevic + AdKDD 2019] (Yahoo) • Latent Cross: Making Use of Context in Recurrent Recommender Systems [Beutel + WSDM 2019] (Google) • Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks [Smirnova + RecSys 2017] (Criteo) • How Airbnb Tells You Will Enjoy Sunset Sailing in Barcelona? Recommendation in a Two-Sided Travel Marketplace [Wu + SIGIR 2017] (Airbnb)
  22. اۀͷڭ܇ 62 • Ad Click Prediction: a View from the

    Trenches [McMahan + KDD 2013] (Google) • Practical Lessons from Predicting Clicks on Ads at Facebookɹ[He + AdKDD 2014] (Facebook) • 150 Successful Machine Learning Models: 6 Lessons Learned at Booking.comɹ[Bernandi + KDD 2019] (Booking.com) • Applying Deep Learning for Airbnb Search [Halder+ KDD 2019] (Airbnb)
  23. ࢿྉͷ঺հʢਪનʣ 63 • Recommender Systems Handbook • RecSys 2020 Tutorial:

    Feature Engineering for Recommender Systems • ʮΦϯϥΠϯ޿ࠂؔ࿈ͷ࿦จΛຊ͘Β͍ࡶʹ঺հ͢ΔAdKDDฤʯ • ʮΦϯϥΠϯ޿ࠂʹ͓͚Δ$53$73ਪఆؔ܎ͷ࿦จΛຊ͘Β͍ࡶʹ঺ հ͢Δʯ • CyberAgent Developers Blog • AWS Recommendation Engine Seminar ࢀՃϨϙʔτʢલ൒ʣ • DeepCTR-Torch
  24. ࢿྉͷ঺հʢςετʣ 64 • Trustworthy Online Controlled Experiments: A Practical Guide

    to A/B Testing • ʮւ֎ͷ༗໊*5اۀͷ"#ςετϒϩά·ͱΊʯ • Innovating Faster on Personalization Algorithms at Netflix Using Interleaving • A Dirty Dozen: Twelve Common Metric Interpretation Pitfalls in Online Controlled Experiments • Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained • Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments
  25. ࢀߟจݙɾࢀߟࢿྉ 65 • Maximizing the Engagement: Exploring New Signals of

    Implicit Feedback in Music Recommendations • Intention Nets: Psychology-Inspired User Choice Behavior Modeling for Next-Basket Prediction • Shopper intent prediction from clickstream e‑commerce data with minimal browsing information • How Airbnb Tells You Will Enjoy Sunset Sailing in Barcelona? Recommendation in a Two-Sided Travel Marketplace • A better clickthrough rate: How Pinterest upgraded everyone’s favorite engagement metric • Real-time User Signal Serving for Feature Engineering • Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction • Deep Interest Network for Click-Through Rate Prediction • Opportunity model for e-commerce recommendation: right product; right time • Temporal-Contextual Recommendation in Real-Time • Learning Efficient Representations of Mouse Movements to Predict User Attention • Time-Aware Prospective Modeling of Users for Online Display Advertising • Understanding Consumer Journey using Attention based Recurrent Neural Networks • User Response Prediction in Online Advertising • Predicting Shopping Behavior with Mixture of RNNs • Beyond Dwell Time: Estimating Document Relevance from Cursor Movements and other Post-click Searcher Behavior • AliMe Assist: An Intelligent Assistant for Creating an Innovative E-commerce Experience • Reinforcement Learning for User Intent Prediction in Customer Service Bots • AntProphet: an Intention Mining System behind Alipay’s Intelligent Customer Service Bot • Predicting Intent Using Activity Logs: How Goal Specificity and Temporal Range Affect User Behavior