Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Web Conference2020 参加報告会

The Web Conference2020 参加報告会

Avatar for Shuhei Goda

Shuhei Goda

April 30, 2020
Tweet

More Decks by Shuhei Goda

Other Decks in Technology

Transcript

  1. ©2020 Wantedly, Inc. 1. Deep Transfer Learning for Search and

    Recommendation ɾݕࡧɾਪનγεςϜʹ͓͚ΔసҠֶशͷԠ༻ͱLinkedinͰͷࣄྫΛ঺հ͍ͯ͠Δ (Tutorial) 2. Clustering and Constructing User Coresets to Accelerate Large- scale Top-K Recommender Systems ɾਪનର৅ͷϢʔβΛᅂ޷ͷ͍ۙ΋ͷಉ࢜ͰΫϥελϦϯά͠, ֤Ϋϥελͷ୅දϕΫτϧΛ
 ɹ ࢖ͬͯۙࣅ࠷ۙ๣୳ࡧΛ͢Δ͜ͱͰਪનγεςϜͷεϐʔυΞοϓΛ࣮ݱ 3. Adaptive Hierarchical Translation-based Sequential Recommendation ɾϢʔβߦಈͷ࣌ܥྻΛߟྀͨ͠Translation-based Reccommendationʹ, ΞΠςϜؒͷ
 ɹ ؔ܎ੑͱ࣌ؒґଘͷuser translation vectorΛՃ͑ͨख๏ΛఏҊ ຊൃදͰ঺հ͢Δ࿦จɾηογϣϯ
  2. ©2020 Wantedly, Inc. Deep Transfer Learning 1. Model Transfer ɹɾsource

    domainsͰͷֶशࡁΈϞσϧΛtarget domainͰ࢖͏ ɹɾfine-tuning (e.g. BERT) ΍ ֤domainͰͷtaskͷmulti-task learning 2. Feature Representation Transfer ɹɾҟͳΔdomainؒͰڞ༗Մೳͳಛ௃Λֶश ɹɾsource domainͰͷֶशࡁΈϞσϧ͔Βͷಛ௃நग़ ΍ domain adaptation 3. Instance Transfer ɹɾsource domain͔Β܇࿅σʔλΛ࣋ͬͯ͘Δ
  3. ©2020 Wantedly, Inc. Deep Transfer Learning for Search and Recommendation

    ݕࡧɾਪનγεςϜͷͲͷ෦෼ͰసҠֶशΛ࢖͏͔ Raw Query Query Understanding User Profile, Activities Raw Document User Understanding Document Understanding Document Retrieval Ranking ←ΑΓྑ͍ཧղͷͨΊʹ, pretrained model
 ͔ΒຒΊࠐΈදݱΛྲྀ༻ (Feature Transfer) ←ຒΊࠐΈۭؒ಺ͰͷީิΞΠςϜͷ֦ு (Feature Transfer) ←ؔ࿈͢ΔαʔϏεͷσʔλ΍ϞσϧΛ
 ྲྀ༻͢Δ(Model Transfer & Instance Transfer)
  4. ©2020 Wantedly, Inc. Case study in Linkedin (1) LinkedinͰ͸ෳ਺ͷϓϩμΫτʹ͓͚ΔϢʔβͷߦಈཤྺ͕औಘͰ͖Δ Job

    Search People Search ଞϢʔβ΁ͷ
 ܨ͕ΓϦΫΤετ ืू΁ͷԠื https://www.linkedin.com/
  5. ©2020 Wantedly, Inc. Case study in Linkedin (2) People Search

    + Careers Job Recommendation ɾ֤υϝΠϯͷਪનλεΫΛղ͍ͯuser, itemͷදݱΛֶश͠, ಛ௃ྔͱͯ͠ར༻ Add embeddings trained with... AUC Lift NDCG@25 Lift Tree Model Baseline
 (no embedding added) 0.00% 0.00% + People Search Embedding (Single-task) 1.25% 0.85% + Multi-task Embedding 1.82% 1.50%
  6. ©2020 Wantedly, Inc. Motivation ྨࣅΞΠςϜͷ୳ࡧͷߴ଎Խ ɾϢʔβʹਪન͢ΔΞΠςϜΛ୳͢ࡍ, PredictionʹؔΘΔ࣮ߦ࣌ؒΛશମతʹ୹͍ͨ͘͠ ɹɾPrediction time =

    Preparation time + Inference time ɹɾطଘͷ୳ࡧΞϧΰϦζϜͰ͸Inferece timeͷ୹ॖʹয఺Λ౰͍ͯͯΔ
 ɾطଘͷख๏Ͱ͸Ϣʔβؒͷؔ܎ੑΛߟྀ͍ͯ͠ͳ͍ͷͰ, ߴ଎Խʹ׆༻͍ͨ͠ ɹɾ࠷ۙ๣୳ࡧ͸೚ҙͷΫΤϦϕΫτϧ͕༩͑ΒΕͨ࣌ʹ͍ۙϕΫτϧΛ୳͢໰୊͕ͩ, 
 ɹɹਪનγεςϜͰ͸ΫΤϦϕΫτϧ͸ϢʔβϕΫτϧͰ΋͋Γ, ϢʔβϕΫτϧ͸ڧ͍
 ɹɹΫϥελϦϯάߏ଄Λ͍࣋ͬͯΔ ɹɾ͍ۙϢʔβ͸ࣅͨؔ৺Λ͍࣋ͬͯΔͨΊ, ୳ࡧ࣌ʹ·ͱΊͯ͠·͑͹ޮ཰తͰ͸?
  7. ©2020 Wantedly, Inc. Framework Overview Preparation Stage: 1. αϒαϯϓϧͨ͠ϢʔβΛΫϥελϦϯά 2.

    άϧʔϓ಺ͷϢʔβΛΧόʔͰ͖ΔΑ͏ͳ
 ෳ਺ͷ୅දϕΫτϧΛϢʔβάϧʔϓຖʹٻΊΔ 3. ΞΠςϜͷ֊૚ܕάϥϑߏ଄Λߏங͠, ɹ֤Ϣʔβάϧʔϓͷ୅දϕΫτϧΛ༻͍ͨ
 ɹgreedyͳάϥϑ୳ࡧʹΑΓ, άϧʔϓຖʹ
 ɹީิΞΠςϜू߹Λऔಘ Prediction Stage: 4. ༧ଌର৅Ϣʔβͷॴଐ͢ΔάϧʔϓΛ୳͠, ɹͦͷάϧʔϓʹରԠ͢ΔީิΞΠςϜू߹ͱ ɹϢʔβϕΫτϧͷ಺ੵΛͱͬͯϥϯΫ෇͚
  8. ©2020 Wantedly, Inc. Motivation ΞΠςϜؒͷؔ࿈ੑͱϢʔβͷߦಈܥྻΛߟྀͨ͠ਪનΛ͍ͨ͠ ɾϢʔβͷߦಈܥྻʹج͍ͮͨਪન (Sequential recommendation) ɹɾe.g. RNN,

    Ϛϧίϑ࿈࠯, CNN, Translated-based Recommendation ɾϢʔβͷߪങܥྻͷཧղʹ͸ΞΠςϜؒͷ૬ิؔ܎ͱ୅ସؔ܎͕ॏཁ ɹɾPCΛߪೖͨ͠௚ޙʹผͷPCΛങ͏͜ͱ͸ͳ͍͕, ͕࣌ؒܦͬͨΒങ͍׵͑ͨ͘ͳΔ
  9. ©2020 Wantedly, Inc. Model Overview Inputs: User Sequences Item Relations

    ิ׬ؔ܎ ୅ସؔ܎ Ouputs: Ϣʔβ͕࣍ʹڵຯΛ࣋ͪͦ͏ͳΞΠςϜ
  10. ©2020 Wantedly, Inc. Item Relations ΞΠςϜ͸ೖΕସΘΓͷස౓͕ߴ͍, ·ͨϢʔβ͔Βߪങ͞ΕΔΞΠςϜ͸εύʔε. ͦ͜ͰΧςΰϦϨϕϧͷؔ܎ΛݟΔ͜ͱʹ͢Δ. ←ҟͳΔΞΠςϜΛ
 ɹߪೖ͍ͯ͠Δ

    ←ΧςΰϦΛݟΔͱ ɹߪങύλʔϯ͕ಉ͡ ΞΠςϜؒͷؔ࿈ੑ → ΧςΰϦϨϕϧͰݟΔ Ϣʔβͷߦಈܥྻ → ΞΠςϜϨϕϧͰݟΔ
  11. ©2020 Wantedly, Inc. Recommendation with HierTrans ←Ϣʔβͷߦಈܥྻ T࣌఺·ͰͷܥྻΛ΋ͱʹ ֤αϒάϥϑ͔Β৘ใΛऔಘ (Head͸GIͱGCΛ


    ݁߹ͤ͞ΔͨΊʹؔ਺) T࣌఺·Ͱͷܥྻͱ
 ΞΠςϜͷؔ܎ੑΛ࢖ͬͯ, ϢʔβͷTranslation vectorΛ ٻΊΔ