Upgrade to Pro — share decks privately, control downloads, hide ads and more …

竟然有人說AI不能拿來做行銷

 竟然有人說AI不能拿來做行銷

Speaker: Johnson
Event: 台大AI社

LINE Developers Taiwan

November 01, 2022
Tweet

More Decks by LINE Developers Taiwan

Other Decks in Technology

Transcript

  1. Hello! - Johnson Wu - CSIE (B01, R05) - Intern

    @ Microsoft Research Asia - Poetry book writer (indirect) - Data Scientist @ LINE Taiwan 2019 @LINE Fukuoka
  2. Our actual thought of Marketing: Content & Experience Social &

    Relationship Commerce & Sales Management Data Advertising & Promotion • Talent Mgt. • Finance Mgt. • Quality Mgt. • CRM • Channels & Partners • Sales Automation & Intelligence • CDP/DMP • Business Intelligence • Data Governance • Customer Exp. • KOL • Community & Review • Content • Experience Mgt. • SEO • Marketing • Advertising • PR Align with user journey!
  3. Look into our problems with… (A)AARRR Funnel ! How to

    let people visit my content? How to let people have action/buy? How to make people come back and buy? How to make people start to buy? How to let people refer new users to your content? Acquisition Activation Retention Revenue Referral 1.25 2.5 3.75 5
  4. Commercial problems with the AARRR Real Cases in LINE Taiwan

    How to recommend OA push to users with limited number of pushes? How to attract most possibly new users to the service? How to value customers and Recommend based on values? How to deliver ad promotion with the best cost-profit balance? How do we understand users for AD targeting? Acquisition Activation Retention and activation AARRR Revenue
  5. What are active users in the service? Where should I

    find the new users? How to connect new users with active users? Acquisition
  6. Users in LINE Correlate users with behaviors of other services

    -*/& 50%": -*/& 
 4)011*/( -*/& .64*$ ? Acquisition 0"
  7. Graph embedding helps Service knowledge graph provides abundant information -

    GraphSAGE (inductive), GCN - Graph Attention network with deepwalk - Semantic matching based: ComplEX - Algorithm choice depends on data traits https://arxiv.org/pdf/1710.10903.pdf https://arxiv.org/pdf/1706.02216.pdf 1 2 1 2 Acquisition
  8. How does graph work? Model predict the probability of the

    “link”. - Example: Graph Attention network with deepwalk 1 2 Acquisition - Users are represented as edge connections between entity with meaningful actions - Users1: article1 read -> 
 shop item2 -> join campaign 3…. - Each node (entity) is aggregated 
 by the weighted sum of neighbors.
  9. New acquired users Case study on LINE POINTS wall :

    how to acquire new users? -*/&10*/54XBMMVTFST OPO-*/&10*/54XBMMVTFST 0 ffi DJBM "DDPVOU GPMMPXJOH TUBUVT ,OPXMFEHF HSBQINPEFMJOH 10*/54 0" $MVTUFSJOH ,OPXMFEHF HSBQIJOGFSFODF 
 $MVTUFSJOH 0" OPO-*/&10*/54 XBMMVTFST NPTUMJLFMZUP KPJO-*/&SFXBSE Compose triples Compose triples Top k similar users
  10. - Flexible and complex decision making - Applied to LINE

    SHOPPING, SPOT, POINTS, MUSIC - Embeddings show clustering effect w.r.t campaign Performance Case study on LINE POINTS : effectiveness and fully data orientation - ≥ 25% of new user growth comes from AI’s decision. - Easily fit for different scenario : 
 How about recommend OA Push message to non-new users? - ≥200% CTR lift Acquisition Activation
  11. Who should receive OAP? How many OAP I should deliver?

    How to evaluate performance? Retention
  12. How to deliver ad promotion with the best cost-profit balance?

    $PVOUPGDBNQBJHO      .POUIPG:FBS             3FDFJWF 0"NFTTBHF 1VSDIBTF JODBNQBJHOQFSJPE &BSO -*/&$0*/10*/54 Boost the Profits with Incentives Retention
  13. User Segments by Uplift High Low Low High Buy if

    treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Retention
  14. User Segments by Uplift EFMJWFSDPTU JODFOUJWFDPTU High Low Low High

    Buy if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Retention
  15. User Segments by Uplift EFMJWFSZDPTU SFWFOVF High Low Low High

    Buy if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Retention
  16. User Segments by Uplift EFMJWFSZDPTU High Low High Low Buy

    if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Retention
  17. User Segments by Uplift  EFMJWFSZDPTU  *ODFOUJWFDPTU  SFWFOVF

    High Low Low High Buy if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Retention
  18. User Segments by Uplift QPTJUJWFVQMJGU OFHBUJWFVQMJGU OPVQMJGU OPVQMJGU Retention High

    Low Low High Buy if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Uplift M odel
  19. User Segments by Uplift Retention High Low Low High Buy

    if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable 3FTQPOTF.PEFM Uplift M odel
  20. User 360 Degree Features Campaign Period Observation Period, 30 days

    Purchase or not Purchase or not Browsing behavior Demographics Purchase history OA engagement Ad event Sticker collections Treatment Group Control Group Retention
  21. User 360 Degree Features Campaign Period Observation Period, 30 days

    Purchase or not Purchase or not Browsing behavior Demographics Purchase history OA engagement Ad event Sticker collections Treatment Group Control Group If the campaign effect is significant  If the sure thing exist  Retention
  22. Meta-learners, S-Learner Campaign Period Observation Period, 30 days Browsing behavior

    Demographics Purchase history OA engagement Ad event Sticker collections Purchase probabilities Treatment Group Control Group $MBTTJ fi FS treatment = 1 treatment = 0 Purchase or not ≈  *OUVJUJWFBOEFBTZUPUSBJO  -PXWBSJBODFBOE fl FYJCMF Retention
  23. Estimate Individual Treatment Effects Campaign Period Observation Period, 30 days

    Browsing behavior Demographics Purchase history OA engagement Ad event Sticker collections $MBTTJ fi FS Purchase probability Purchase probability treatment = 1 treatment = 0 candidate Retention
  24. Estimate Individual Treatment Effects Campaign Period Observation Period, 30 days

    Browsing behavior Demographics Purchase history OA engagement Ad event Sticker collections $MBTTJ fi FS treatment = 1 treatment = 0 Uplift Purchase probability candidate Retention
  25. Uplifts by Deciles 6QMJGU      

             The Persuadable Sure Thing/Lost Cause Sleeping Dog Retention
  26. Reduce Delivery Cost and Mute Rate in the Future 6TFS-JTU

    %JTDPVOU 6TFS-JTU $PJO1PJOU #BDL 6TFS-JTU -VDLZESBX Day 3 Day 2 Day 1 Campaign Period 4UJDLFS %BUB %FW Retention
  27. Know Your Customers F R M Historical Unknown Time Now

    From Buying Profile Retention Revenue
  28. Measure Your Customer Lifetime Value From RFM to CLV Past

    Now Future Measure Former Value Predict Future Value RFM CLV Retention Revenue
  29. Measure Your Customer Lifetime Value From RFM to CLV Past

    Now Future RFM CLV Measure Former Value Predict Retention Revenue
  30. Measure Former Value - RFM Modeling R F M RFM

    Segments Historical data Clustering Retention Revenue
  31. Measure Former Value - RFM Modeling R F M Historical

    data Clustering Retention Revenue
  32. Measure Your Customer Lifetime Value From RFM to CLV Past

    Now Future RFM CLV Measure Former Value Predict Future Value Retention Revenue
  33. Predict Future Value - CLV Modeling Demographic Browsing history Buying

    pro fi le OA interaction Feature Engineering Modeling Tags CLV High ↑ CLV Median CLV Low ↓ { > 50% High valued customers <= 50% Medium valued customers 0, Churned customers 450 days 180 days Now Cascade model Retention Revenue
  34. Predict Future Value - CLV Modeling Feature Engineering Modeling Tags

    CLV High ↑ CLV Median CLV Low ↓ { > 50% High valued customers <= 50% Medium valued customers 0, Churned customers 450 days 180 days Now User Embedding Demographic Browsing history Buying pro Retention Revenue
  35. Predict Future Value - CLV Modeling User Embedding Time of

    View Content Views from customers AD 1 AD n Product 1 Article 1 Article n Product n … … … Context window Retention Revenue
  36. Explicit and explainable user segment R↑F↑M↑ R↑F↑M↓ R↑F↓M↑ R↑F↓M↓ R↓F↑M↑

    R↓F↑M↓ R↓F↓M↑ R↓F↓M↓ ࠷ۙ༗ိత౔߽ৗ٬ ࠷ۙ༗ိతখࢿৗ٬ ࠷ۙ༗ိత౔߽ك٬ ࠷ۙ༗ိతখࢿك٬ ফࣦత౔߽ৗ٬ ফࣦతখࢿৗ٬ ফࣦత౔߽ك٬ ফࣦతখࢿك٬ CLV Low CLV Median CLV High ະိߴফඅ ະိதফඅ ະိෆফඅ RFM CLV Deliverables: Tags Retention Revenue
  37. R↑F↑M↑ CLV Low Retain ! ೗Կར༻໛ܕඪតʁ RFM + CLV RFM

    CLV 最近有來的⼟豪常客 未來不消費 Retention Revenue
  38. Any model that could handle everything? A good start of

    user understanding ! #MBDLCPY Who among them are more likely to come? Which content is more likely be click by whom? Who among them are more likely persuadable? How to estimate user values? How to refer friends of users to us?
  39. Foundation models Pertaining on user behaviors, which come from our

    family services log! Click prediction User segment prediction Sales/revenue prediction
  40. Combined with MLOps: 
 Scalable Automated MarTech Solution Data Source

    %BUB 8BSFIPVTF Data Integration 'FBUVSF4UPSF Application "DRVJSFOFXVTFS "%5BSHFUJOH #PPTUSFWFOVF *NQSPWF3FUFOUJPO … ,(4PMVUJPO1JQFMJOF $-74PMVUJPO1JQFMJOF 6QMJGU 
 4PMVUJPO1JQFMJOF AI Solution 6TFSTFHNFOUBUJPO 
 4PMVUJPO1JQFMJOF … LINE Family Services Demographic Log
  41. To sum up Swifter Scalable and Flexible Solution Faster Speed

    of Delivery Data-oriented Decision Making AI solves (almost) every Martech problem Developer Marketer YOU
  42. Thank you Closing the Distance Official Account Ads 圖⽚來源︓https://hub.line.me/ 如何推薦用戶會感興趣的商家?

    如何確保新聞品質? 如何判斷假新聞? 如何判斷潛在的訂閱制流失戶?
  43. Thank you AI-enable Applications Business Intelligence Data Dev LINE Family

    Services LINE TODAY LINE SHOPPING LINE SPOT LINE MUSIC LINE Sticker LINE VOOM LINE Reward Official Account Fact Checker LINE HELP TW LINE Travel Ads 獨立的資料工程部門,提供資料科學解決方案 LINE TODAY
  44. Thank you Data Dev LINE Family Services LINE SHOPPING LINE

    SPOT LINE MUSIC LINE Sticker LINE VOOM LINE Reward Fact Checker LINE HELP TW LINE Travel NLP Knowledg e Graph Uplift Modeling NER Classifier Duplication Detector Auto completion Keyword Extraction Related Search Text Generation User Tagging Data Analytics Recom- mendation CLV 從報表、分析洞見到預測模型 LINE TODAY
  45. Thank you 成員組成 • Build and optimize da ta pipeline

    architectur e • Assemble large, com plex data sets that m eet requirements Data Engineer Data Analyst Big data infra, SQL, ET L, message queuing • Interpret data, analyz e results using statisti cal techniques • Identify, analyze, and interpret trends or pat terns in complex data sets Statistics, Data Visualiz ation, Business Knowle dge SKILL RESPONSIBILITY • Select appropriate da tasets and data repre sentation methods • Research and imple ment appropriate ML algorithms Data Scientist Machine learning, deep learning, CV, NLP, Spe ech ML Svc Engineer • Build and scale mach ine learning infrastruc ture • Monitor model perfor mance System infrastructure d esign, DevOps
  46. Thank you Subtitle 成員組成 • Build and optimize da ta

    pipeline architectur e • Assemble large, com plex data sets that m eet requirements Data Engineer Data Analyst Big data infra, SQL, ET L, message queuing • Interpret data, analyz e results using statisti cal techniques • Identify, analyze, and interpret trends or pat terns in complex data sets Statistics, Data Visualiz ation, Business Knowle dge SKILL RESPONSIBILITY Pipeline Biz • Select appropriate da tasets and data repre sentation methods • Research and imple ment appropriate ML algorithms Data Scientist Machine learning, deep learning, CV, NLP, Spe ech Model ML Svc Engineer • Build and scale mach ine learning infrastruc ture • Monitor model perfor mance System infrastructure d esign, DevOps Service