Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Enhanced EC Recommendations: Trustworthy Valida...
Search
LINE Developers Taiwan
PRO
September 23, 2024
Technology
0
32
Enhanced EC Recommendations: Trustworthy Validation with Large Language Models for Two-Tower Model
Event: iThome Hello World Dev Conference
Speaker: Dan Chen
LINE Developers Taiwan
PRO
September 23, 2024
Tweet
Share
More Decks by LINE Developers Taiwan
See All by LINE Developers Taiwan
LINE 實習分享 & 國際黑客松參賽分享
line_developers_tw
PRO
0
19
在 GCP 運用 Parse 全家餐管理那堆 AI 應用的資料
line_developers_tw
PRO
0
22
40歲的我會給20歲的自己,關於軟體開發的7個建議
line_developers_tw
PRO
0
7.5k
從零到一:轉碼仔的實習攻略
line_developers_tw
PRO
0
29
如何在團隊發揮數據影響力: 以電商資料科學家為例
line_developers_tw
PRO
1
44
做Data超讚的 誰懂?
line_developers_tw
PRO
0
29
iOS Live Activity: Opportunities & Challenges
line_developers_tw
PRO
1
120
掌握 Feature Toggle 與 OpenFeature 規範
line_developers_tw
PRO
0
230
用 AI 和 LINE Bot 簡化生活:讓圖片告訴你何時該忙!-- LINE 工作坊
line_developers_tw
PRO
0
760
Other Decks in Technology
See All in Technology
MC906491 を見据えた Microsoft Entra Connect アップグレード対応
tamaiyutaro
1
460
Larkご案内資料
customercloud
PRO
0
600
現場で役立つAPIデザイン
nagix
27
9.2k
データ基盤の成長を加速させる:アイスタイルにおける挑戦と教訓
tsuda7
3
640
ハッキングの世界に迫る~攻撃者の思考で考えるセキュリティ~
nomizone
10
4.1k
自動と手動の両輪で開発するデータクレンジング
estie
2
210
開発スピードは上がっている…品質はどうする? スピードと品質を両立させるためのプロダクト開発の進め方とは #DevSumi #DevSumiB / Agile And Quality
nihonbuson
1
860
Data-centric AI入門第6章:Data-centric AIの実践例
x_ttyszk
1
350
サーバーレスアーキテクチャと生成AIの融合 / Serverless Meets Generative AI
_kensh
12
2.9k
スクラムのイテレーションを導入してチームの雰囲気がより良くなった話
eccyun
0
100
エンジニアのためのドキュメント力基礎講座〜構造化思考から始めよう〜(2025/02/15jbug広島#15発表資料)
yasuoyasuo
11
4.6k
オブザーバビリティの観点でみるAWS / AWS from observability perspective
ymotongpoo
1
250
Featured
See All Featured
Side Projects
sachag
452
42k
Designing on Purpose - Digital PM Summit 2013
jponch
117
7.1k
Mobile First: as difficult as doing things right
swwweet
223
9.3k
Done Done
chrislema
182
16k
GitHub's CSS Performance
jonrohan
1030
460k
Music & Morning Musume
bryan
46
6.3k
Building Your Own Lightsaber
phodgson
104
6.2k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
232
17k
Code Review Best Practice
trishagee
66
17k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
A Philosophy of Restraint
colly
203
16k
Transcript
None
Enhanced EC Recommendations: Trustworthy Validation with Large Language Models for
Two-Tower Model EC Data Dev / Data Scientists Dan Chen
Dan LINE Taiwan EC Dev - Data Scientis Work Experience
Side Project
01 02 03 04 Evaluation Framework Offline & Online Evaluation
LLM on Recommendation What is Trustworthy 05 Q&A CONTENT
Why it’s so important 01 What is Trustworthy
Element of trustworthy 特點項目文字 特點項目 Trustworthy 特點項目文字 特點項目 特點項目文字 特點項目
Four Perspective 特點項目文字 特點項目 Trustworthy Recommendation 特點項目文字 特點項目 特點項目文字 特點項目
Data Preparation Data Representation Recommendation Generation Performance Evaluation
How to Correctly Evaluate AI 02 Evaluation Framework
Two - Stage Recommendation system Brickmaster Scalable Scenario-wise KPI -
Oriented Trustworthy
How to truly comprehensive understand performance Evaluation Framework (1/2)
How to truly comprehensive understand performance Evaluation Framework (1/2)
How to Correctly Evaluate AI 03 Offline & Online Evaluation
Key point to show how your algorithms can contribute to
your business Offline Evaluation
Key point to show how your algorithms can contribute to
your business Online Evaluation
Avoid pitfalls In Practice If experiment isn’t’ significant ?? Sample
ratio mismatch ?? Novelty effect ?? Key point to show how your algorithms can contribute to your business A/B test
Case – EC Shop recommendation
04 LLM On Recommendation
Recommendation with LLM - Feature Engineering: Text embedding generation -
How to evaluate embedding (probing): RankMe / α-ReQ Metrincs
Recommendation with LLM - Feature Engineering: Text embedding generation -
How to evaluate embedding (probing): RankMe / α-ReQ Metrincs
Evaluate & Challenge 05 Conclusion
Conclusion Business Value OpenAI, Claude, Gemini XGBoost or OpenSource 來源:https://zh.wikipedia.org/zh-
tw/%E7%BE%8E%E5%9C%8B%E9%9A%8A%E9%95%B72%EF%BC%9A%E9%85%B7%E5%AF%9 2%E6%88%B0%E5%A3%AB 來源:https://images.app.goo.gl/HCygtJVtoPaU2KgX6
Conclusion & Challenge 1. Data Quality 2. Multiple – Metrics
evaluation 3. Conduct A/B test Experiment 4. Human Perception Evaluation Challenge
Q&A 聯絡資訊 (Linkedin – Dan Chen)
None
None